These options allows us to close the search indices if there is no activity within a specified interval. Once we reach this limit then clouseau will close the oldest index (in terms of when its opened). ![]() By default this is set to 200, and for many large deployments where the customers have many active indices, we had to significantly increase this number(in the above example it's set to 15000). As clouseau uses Lucene, and each index opened by lucene has an overhead (mainly heap), and if we don't close the indices then the JVM will run out of memory and hence this option to limit the number of opened indices. This option allows to specify the maximum number indices opened at a given point of time. This option allows the JVM to set initial memory when the JVM started (clouseau start) to avoid dynamic heap resizing and lags. And if set, keep it at 80% of max_memory and can't be more than max_memory. This setting is optional and is recommended to set in the cases of higher maximum heap size (> 8GB). But be cautious when going over the one third of the available memory as it could result in less memory available for erlang runtime and the OS. So if the nodes on cluster have 30GB of memory available, then limit max_memory setting to 10GB and if that's not enough and the user workload still requires more memory then we may increase it to 15GB (50% of available). The recommendation is to keep max_memory to maximum of one third of the available memory and to never allocate more than 50% of the total available memory. The amount of heap required usually correlates with the max_indexes_open settings, so if we want to have higher number of indices opened then we need higher value set for max_memory. ![]() The amount of heap usage depends on the number of active (opened) search indices and also the search load (sorting etc). By default this is set to 2GB and is recommended to tune it based on the the work load. ![]() max_memoryĪs you know, clouseau's run time is Java and max_memory option allows us to tune the maximum heap size for the JVM. In the following sections, will go over these options and explain the significance of each one of them and provide some guidelines on how to tune them. Tweaking these clouseau options, by updating role file corresponding to the cluster, would enable us to handle the various work load scenarios and allow us to better utilize the available hardware resources.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |