64 bit and very large heap size

Hi,
I want to run Solr on Tomcat.
My index is very large >70m documents ~10 gigs on disk.
My machine is a windows 2008 server 64bit with 8 cores and 64 gigs of ram.

I want to use the largest heap possible, what do u recommend the optimal vm args i should use?

Comments

Hi,
Start by reading this: http://www.tikalk.com/java/performance-tunning-case-study

Its mainly for Unix, but several topics are relevant.

Will get back to you later, promise :)

10x man,
Can't wait
When are you coming to shultz? on me (:

Which JVM version are you using?
How many concurrent clients (doing searches) are you expecting?

Assuming you want the largest heap possible, then you can allocate almost 64GB of memory minus the space allocated for PermGen (non-heap), the OS itself, Apache (if you are running one) etc etc.

Your choice of heap allocation can not be decoupled from the selection of the GC algorithm you are going to use. Frankly, I never run a JVM with more then 16GB of memory (which was also a Unix machien and not windows) and it would be irresponsible to just provide you with JVM parameters and assume that everything would work.

What I do suggest is that you start with a baseline, then run some tests, comparing the performance (using jstat, jprofler,jconsole or whatever) under different scenario's and using this iterative process reach a manageable heap size.

Baseline:
-Xms40g -Xmx40g XX:PermSize=1g -XX:NewSize=8g -XX:MaxNewSize=8g

I have no clue if this would even work on your machine, try it :)

BTW, did you think about Tomcat clustering?

thanks,
I will try it.
Haven't considered tomcat clustering, not sure if its suitable with solr, Will look it up though.

See this:
http://connections.blackboard.com/files/a8760b65e7/BBWorld09_-_Tuning_th...

They are against tomcat clustering. Excellent presentation.