Odi's astoundingly incomplete notes

New entries | Code

Webapp scalability

While researching about the scalability of the Wicket framework, I came across what seems like a widespread myth. In a couple of mailinglists and forums people were basically saying: "Scalability is basically limited by how much data is stored in a session. But memory is cheap, so don't worry". Sad, that this just slightly misses the point.

Scalability is limited by 4 things: Real applications typically hit only one of these limits. If we just look at the web framework, then the DB is out of our focus and we assume that this is not the bottleneck. If you're not doing video streaming then network bandwidth is not likely to be an issue for you. And if you're doing video streaming you better have plenty of spare bandwidth anyway. If you are not doing file serving (or again streaming) disk throughput will also not be an issue. What remains is CPU and memory. If CPU is your limit (the webapp does number crunching), then there is not much you can do: optimize your algorithms. So the only really interesting limiting factor is memory.

Now in Java we have a garbage collected heap. So memory is not completely independent from CPU. GC can consume a fair amount of CPU and it causes latency. So it is really important that the application does as little garbage collection as possible. In a generational GC heap (Concurrent Mark and Sweep - CMS) there is also cheap and fast garbage collection of the relatively small Eden space (the youngest generation) and expensive and slow garbage collection of the Old generation (the bulk of the heap). It's key that you prevent the big collections from happening. They really hurt. Rather have 10'000 small collections instead of 1 big one. That means that you should not "churn" objects, be conservative with what you allocate. And it means that the ones that you allocate temporarily (during a request or shorter) should have the shortest possible lifetime. Each object that survives the Eden collection is a performance killer: it has to be moved to the next generation and will have to be collected from there eventually by a big collection!

So if each request produces 1 MB of garbage and your Eden space is 100 MB, each 100 requests will cause a small collection. Fine if these are the only requests on the system. The requests are over, so all objects are unreferenced and can be collected. The collection will be quick and efficient.
If you now make 200 requests in parallel, the Eden space will be full half-way through the time. If those objects are still referenced now they can not be collected and will be moved to the next generation and you have hit the scalability limit: the Old generation will quickly fill up and big collections will have to run, comsuming CPU and causing latency. If however most of these objects are no longer referenced, they can be collected quickly and your webapp still scales.

What about session state now? Session state doesn't change much usually. So it is more or less constant. Its objects will live in the Old (or even permanent) generation. It's certainly a bad idea to constantly add and remove objects to/from the session, because these objects will have to be collected in the Old generation by a big collection. Session state is also really small usually. Just a few KB. Do the maths: a 2 GB heap can hold 20'000 sessions of 100 KB. That's more than enough and will not be the bottleneck. Of course don't store huge data in sessions.

To sum up this posting: if you want a scalable Java webapp: For frameworks that means: Java objects that are infamous for a lot of overhead in terms of object allocation:
posted on 2009-03-27 01:03 UTC in Code | 1 comments | permalink
All the GC algorythms SUN/Oracle's JVM supports are generational, not just CMS.

CMS is able to continue application-execution while the old-gen is collected at the expense of throughput.
ParallellOldGC will stop all threads during a GC cycles -> longer pauses, but has lower overhead in general.

- Clemens