Reducing Memory Usage

We chose the store format because it reads and writes data directly in a binary file. This easy and quick solution is ideal for the initial idea of covering a period of 5 to maximum 10 minutes in 3 minute intervals.   In the meantime, we use these stores not only as short term memory, but also to save data for trend analysis spanning several weeks, which can lead the several megabytes of data in storage, even reaching hundreds of megabytes at times. It doesn’t really make sense to read this data every few minutes and to save it to the hard disk. In version 3.6 we are introducing a new, more efficient store format. Based on a history_store file with 1001 calls (the points in time when the filer fetches the data) that we received from one of our clients, were able to reduce the usage on the hard disk alone from 11 MiB to 217 KiB. The missing bytes are stored in the sub directory calls in the store directory, available when needed. In percentages, with this new store format for performance and history checks, we are able to reduce the memory usage to 2% of the initial requirement, making corrupt store files a thing of the past. This new store format is available for testing in version 3.6.0_01. We do not recommend this version for production use. For migration all existing store files should be deleted (or moved to be safe). Important store files (history) can be converted to the new format using storemigrator.pl.