90% of Devs Get Java Serverless WRONG (How to Fix It)

Introduction to Java's Popularity and AWS Lambda Support

• Java is one of the most popular programming languages, ranking as the second or third most popular, with its popularity determined by factors such as the number of job postings, GitHub repositories, and Stack Overflow questions 42s.
• On AWS Lambda, Amazon has its own Java distribution called Amazon Corretto, which provides security patches and support, including long-term support for Java 8 until 2030, and currently supports Java versions from 8 to 21 2m6s.
• Despite Java being a fast and mature programming language with a huge community and open-source projects like Spring, its adoption on AWS Lambda is relatively low, with single-digit percent usage, compared to other languages like Python and NodeJS 4m6s.

Challenges of Java on AWS Lambda

• The two main issues with Java on AWS Lambda are cold start times or latency startup times, and memory footprint, which is a cost factor, and these will be the focus of discussion 6m17s.

Overview of a Serverless Application Architecture

• A simplified example of a serverless application is presented, consisting of an API Gateway, two Lambda functions for creating and getting products, and a serverless database like DynamoDB, which is preferred over managed relational databases due to certain challenges 8m20s.
• The Lambda function receives a JSON body from the API Gateway, which contains query parameters, path parameters, body, and headers, and this JSON is deserialized into a Java object, as shown in the example of a get product by ID Lambda function 10m30s.
• The lambda function has only one method, "handle request", which gets a JSON serialized into an API gateway proxy request event, and from this event, the parameter ID can be obtained to retrieve a product from DynamoDB, with the output being a proxy response event containing the HTTP code, message, and the serialized product as the outcome 10s.

Understanding Cold Starts in AWS Lambda

• The biggest challenge in this process is the "cold start", which occurs because Lambda implements a function-as-a-service paradigm, where the execution environment can only handle one request at a time and does not scale horizontally like Tomcat, requiring a new environment to be started when there are not enough available 2m6s.
• When a new environment is needed, AWS is responsible for starting it, which can happen when a lambda function is invoked for the first time, updated, or redeployed, and the existing environments are destroyed and replaced with new ones, and this process can also occur when the number of requests changes, requiring more or fewer environments 2m6s.
• The environments are not constant and can be shut down by AWS due to cost-saving reasons, security patches, or other reasons, and this can happen within a few hours, resulting in the need to start new environments, and this startup process is referred to as a "cold start" 2m6s.
• The cold start process involves starting a new execution environment, which can lead to latency, and AWS tries to minimize this latency by starting new environments in parallel when the number of requests increases, but the exact timing of when environments are shut down or started is not specified by AWS 2m6s.

Lambda Execution Environment and Java Runtime Initialization

• When deploying a lambda function, the code is uploaded to S3 on Amazon, and then downloaded to an execution environment, such as Firecracker microVM, which manages the environment, and if Java is selected, the execution environment starts the Java runtime, specifically the Corretto version 10s.
• The static initializer code block is executed, which includes everything in the static initializer block of the lambda function and everything reachable from it, such as class loading, runtime dependency injection, and just-in-time compilation, and if using a framework like Spring, annotation processing is also performed 1m5s.
• The last step is the invocation of the handler method, which is the function code itself, and the first three steps are referred to as the start or start-up time of the application, while the last step is called the warm start, which is the execution of the function 2m6s.

Cold Start vs Warm Start Performance

• The warm start is only the execution of the function, and it occurs when the warm container is already present, whereas the cold start requires the execution of all three previous steps, and this has implications for performance 3m15s.
• A test was conducted with a lambda function allocated 1 gig of memory, which derives half of a virtual CPU, and the application used an HTTP client to talk to Dynamob, with a size of 14 megabytes, and certain Java compilation options were used, such as disabling tiered compilation 4m20s.
• The test involved a stress test for one hour, with 100 cold starts and 100,000 warm starts, and the results showed that the warm start times were relatively quick, with a P90 value of around 7 milliseconds, which is a good performance indicator 6m30s.
• The cold start for Java can take around 3 seconds, but it does not happen frequently, with less than 1% of invocations resulting in a cold start, however, this can increase if an application uses multiple lambda functions 10s.
• Generally, 2 to 3% of invocations can result in a cold start, which can have a business impact, especially for public-facing applications, as studies have shown that more than 1 second of delay can cause users to drop off 1m42s.

Optimization Techniques: SnapStart and Gravium

• To optimize cold starts, two techniques can be used: SnapStart and Gravium, with SnapStart being AWS's own technology, developed to improve startup performance for Java runtimes, and now also available for Python and .NET 2m6s.
• SnapStart is a fully managed technology that works by dividing the deployment phase into two phases, with the first phase involving the execution of the cold start phase of the application, and the second phase involving the restoration of a snapshot of the application, which contains everything loaded by the JVM 3m30s.
• The snapshot created by SnapStart includes the complete state of the microVM, including the Linux operating system, JVM, and loaded classes, and during the invocation phase, this snapshot is restored, allowing for quicker execution and potentially reducing the cold start time 5m20s.
• SnapStart is currently only available for managed runtimes like Java, Python, and .NET, and does not support Docker container images, which can be used as an alternative way to deploy applications on AWS Lambda 4m40s.

Implementing SnapStart and Priming Techniques

• The decision to use a snap start in a lambda function is made by adding a single line of code to the infrastructure code, which is a true or false flag, and this flag determines whether to use snap start or not 10s.
• The technique of priming can be used with snap start, which involves loading and pre-initializing as much as possible into the snapshot, so that the restore is better, and this can be done by loading classes and pre-initializing stuff that will become part of the snapshot 1m6s.
• Java loads classes lazily and partially, and some things like initializing an Apache HTTP client can take around half a second, which can be pre-initialized to become part of the snapshot 1m42s.
• When talking to DynamoDB, JSON marshalling or unmarshalling is involved, which can be pre-initialized by creating a new object mapper, and this can save around 300-500 milliseconds of execution time 2m6s.
• The pre-snapshot hook is an important method that can be used to implement additional logic to pre-initialize stuff that will become part of the snapshot, and priming is a way to fake pre-initializing the stuff to speed up things 3m6s.
• Even in the warm start phase, the first execution of a function can be slow because classes need to be loaded, and this can be dealt with by pre-initializing the classes and logic that will be used, such as the get product by ID logic in DynamoDB 4m6s.

Advanced SnapStart Implementation and Preloading

• To implement Java serverless correctly, it is necessary to include an additional dependency called orc crack, which provides the possibility to implement the resources interface, and register the lambda function by copying a specific line of code, allowing for the loading of necessary resources before the checkpoint, which becomes part of the snapshot 10s.
• The beforeCheckpoint method from the resource interface can be used to preload necessary classes and initialize resources, such as executing an HTTP client to load required classes, which then become part of the snapshot and survive the snapshot and restore process, eliminating the need to rewrite code for connections to services like DynamoDB 1m20s.
• SnapStart is a feature that can help reduce cold starts by preloading classes and resources, and it requires some knowledge of what can be primed, with a SnapStart priming guide available from AWS that describes how to preload classes that will be loaded lazily 2m30s.
• Measurements have shown that using SnapStart without priming can reduce cold starts from 3.2 seconds to 2 seconds, and with priming, it can be reduced further to 1 second, with the P90 percentile being a good indicator of performance, and the max value of warm starts also decreasing significantly 3m40s.

Performance Impact of SnapStart and Configuration Tuning

• The warm start is not improved by SnapStart, but the max value can be decreased by preinitializing resources, and the same result can be achieved without SnapStart by using a static initializer block, but SnapStart is necessary for improving cold starts 5m10s.
• The configuration of SnapStart can be tuned by adjusting the lambda memory, with less memory potentially resulting in less cost but also less CPU, requiring experimentation to find the optimal balance 6m50s.

Java Performance Optimization in Serverless Environments

• To improve Java serverless performance, it is possible to increase the memory allocated to the application, up to 8 GB, after which a second core is added, but this may not always be beneficial and could increase costs 10s.
• Java compilation options are important to test, and using a specific option can help optimize performance by not aggressively optimizing lambda functions, which are short-lived and may not benefit from the default tiered compilation 42s.
• The choice of HTTP client implementation can also impact performance, with options including Apache, AWS native, and URL connection, and priming can help preload dependencies and reduce the difference in performance between implementations 1m30s.
• The architecture used, such as ARM, can also affect performance and cost, with ARM being cheaper but potentially degrading performance, and the use of priming techniques can help mitigate this 2m6s.
• The size of the application is crucial in serverless environments, particularly when using Tomcat, as larger applications with more dependencies can result in longer cold start times, even with snap start and priming 3m20s.
• It is essential to carefully manage dependencies and only include necessary ones, as test dependencies can increase the application size and lead to longer cold start times and increased costs 4m30s.

AWS Lambda Cache and Its Impact on Performance

• The performance of serverless applications can be affected by the snapshot storage mechanism used by AWS, which stores snapshots in a low-latency cache across three availability zones, and the cache is chunked into 512 kilobyte pieces 6m0s.
• The frequency of execution and the size of the image are factors that affect the performance of a lambda function, with smaller and more frequently called functions being favored, and this is due to the way AWS traces the execution path and preloads chunks asynchronously 10s.
• Infrequently accessed data will go to L2 cache or even to S3, and it's worth noting that the cache becomes faster with time as it is filled with chunks of the function, but it cannot be configured 1m30s.
• The cache's performance improves over time as it is filled with chunks of the function, and the more frequently a lambda function is executed, the more cache is filled, resulting in quicker execution 2m6s.
• To accurately measure the benefits of the cache, it's recommended to measure until the end, as the first few executions may be slower, but the subsequent ones will be quicker, and this is especially important when using snap start and priming 4m10s.

Performance Measurement and Profiling Tools

• The benefits of the cache can be seen in the comparison of snap start with and without priming, where the last 70 executions were quicker than the first 30, resulting in a significant reduction in execution time, from 1.2 seconds to 650 milliseconds 5m20s.
• AWS has released an asynchronous profiler that can be used to look into what's happening in a lambda function, which is based on an open-source asynchronous profiler, and this can be useful for optimizing lambda function performance 8m30s.
• To optimize Java serverless functions, flame graphs can be used to identify where time is spent and what is loaded when, allowing for potential gains by preloading and priming certain components 10s.

Optimization Through Flame Graphs and Profiling

• By analyzing the flame graph, it is possible to extend code and prime certain libraries, such as the lambda serializer, to improve performance, with one example achieving a 20% improvement in call start time 42s.
• The use of Java SnapStart can help improve performance, but it is currently free only for Java, and AWS may introduce pricing for cache and restore phases in the future, similar to what has been done for Python and .NET 2m6s.

Limitations and Considerations of SnapStart

• When using SnapStart, it is essential to consider uniqueness and avoid using time-based caches or static initializer blocks that rely on current time, as the restore time from the snapshot can be unpredictable 2m6s.
• There are limitations to using SnapStart, including the need to secure and store snapshots in all availability zones, which can take two to two and a half minutes for a lambda function, and the snapshot being deleted if the function is not invoked for 14 days 4m0s.
• Deploying multiple lambda functions in parallel with SnapStart can help mitigate the deployment time penalty, but it may still be beneficial to avoid using SnapStart for staging environments where quick testing is prioritized over performance 5m0s.

Introduction to Gravium and Native Image Compilation

• Java developers currently face limitations due to AWS protecting its storage, but this limitation may be removed if pricing is introduced, allowing developers to pay for storage 10s.
• Gravium provides ahead-of-time compilation, creating a closed-world native image with all reachable methods, classes, and fields, which is beneficial for serverless applications due to its low memory footprint, small packaging, and fast startup speed 42s.
• Native executables are smaller and require less memory because they don't need a just-in-time compiler, and AWS allows the deployment of custom runtimes, including Gravium images, which can be packaged as a zip file with a native image 1m6s.
• Gravium offers predictable and low cold start times, as well as low warm start times, but requires dependencies to be Gravium-ready, and provides a page listing compatible frameworks and dependencies, such as Quark, Skeleton, Spring, and Micronaut 2m6s.
• Using Gravium requires declaring classes in advance, especially those used for reflection, and provides a possibility to use an assistant tracing agent to find and generate the necessary code, but may still require manual declaration of missing classes 3m6s.

Challenges and Developer Experience with Gravium

• Gravium is a powerful technology with a lot of potential for improving cold start times, memory footprint, and startup speed, but managing the execution pipeline to create a Gravium native image can be challenging, which is why some developers prefer managed solutions like SnapStart 4m6s.
• Building a Java serverless application requires a pipeline, which can be costly and time-consuming, with a single lambda function potentially requiring 6 GB of memory and taking up to three minutes to build a native image, resulting in higher costs 10s.
• The developer experience is affected by the build time, with a three-minute build time comparable to snap starts, and the need to carefully watch for runtime errors that can occur when updating dependencies 42s.

Comparing SnapStart and Gravium for Java Serverless

• Snap start and GraalVM are both valid technologies, but snap start is fully managed and may be a better starting point for those who can accept higher cold and warm start times, while GraalVM is more suitable for applications that require millisecond precision 1m30s.
• GraalVM announced its detaching from the Java ecosystem train, focusing on non-Java languages, which has raised questions about the future support of GraalVM native image, and an alternative is Project Loom, which aims to improve start time, peak performance, and footprint of Java programs 2m6s.
• Project Loom is not the same as GraalVM native image and has a worse performance due to the use of IoT cache and glass data sharing cache, leaving the future of GraalVM native image uncertain 2m40s.

Dependency Management and Cold Start Optimization

• The size of dependencies can increase with new versions, resulting in longer cold start times, and it is essential to monitor and optimize dependencies to minimize cold start times 4m10s.

New AWS Services and Future Directions

• AWS has released new services, such as Aurora DSQL, a PostgreSQL-compatible database that can help solve connection management issues in serverless applications, and its performance will be measured and compared in future tests 6m0s.

Conclusion and Recommendations for Java Serverless Adoption

• The key takeaway is that it is possible to significantly reduce cold start times, and exploring options like snap start can provide numerous opportunities, with the potential for further improvement to achieve start times below 1 second, even if it only happens 2-3% of the time 10s.
• Having a solid Java knowledge base can make it viable for companies to adopt a serverless approach or use Java on AWS Lambda without excessive concern 42s.
• Java developers can consider using serverless options, such as AWS Lambda, without fear, as long as they have a good understanding of Java, which can help mitigate potential issues 2m6s.