Memory consumption of big calendar
July 29, 2008
How much memory is consumed by the calendar ? I hope, that this post helps to answer the question.
I will use my favourite big calendar.ics file. Some basic facts about calendar:
slusny@turret:~/gsoc/massif logs$ ls -lh calendar.ics
-rw-r--r-- 1 slusny slusny 813K 2008-07-28 14:15 calendar.ics
slusny@turret:~/gsoc/massif logs$ grep VEVENT calendar.ics | wc -l
2550
Saying this, I consider the calendar as a big one. I prepared simple program, that just opens the calendar and parses the ICS file, similarly to file backend. Valgrind’s massif gave me this graph
MB
7.192^ ,,:. .
| . . @ @#:: :.
| . @@: : @ @#:: ::
| @@: @@: : @ @#:: ::
| ,, @@: @@: : @ @#:: ::@
| . @@@ @@: @@: : @ @#:: ::@
| .. : @@@ @@: @@: : @ @#:: ::@
| .@ :: : @@@ @@: @@: : @ @#:: ::@
| ,@:@ :: : @@@ @@: @@: : @ @#:: ::@
| ,@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@:
| ,: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@:
| , :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@:
| @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@:
| ., @ @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@:
| . ,:@ @ @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@:.
| .@: @:@ @ @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@::
| , @:@: @:@ @ @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@::
| ,:@@ @:@: @:@ @ @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@::
| , @ @:@@ @:@: @:@ @ @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@::
| @@ @ @:@@ @:@: @:@ @ @@@ :@: @@ @@:@ :: : @@@ @@: @@: : @ @#:: ::@::
0 +----------------------------------------------------------------------->Mi
0 287.6
So, the calendar consumes more thatn 7 MB. I ran the program again without valgrind and looked at process
with smem.pl script. This script parses /proc/$pid/smaps into something more useful
VMSIZE: 18136 kb
RSS: 10784 kb total
2864 kb shared
168 kb private clean
7752 kb private dirty
PRIVATE MAPPINGS
vmsize rss clean rss dirty file
7148 kb 0 kb 7148 kb [heap]
40 kb 0 kb 40 kb /usr/local/lib/libORBit-2.so.0.1.0
44 kb 0 kb 40 kb /home/slusny/gsoc/evolution-data-server/calendar/libecal/.lib
88 kb 0 kb 36 kb [stack]
…
The heap is shown as smaller here (who knows why ?). However, the calendar is quite big, I think. Perhaps iwe should not hold all events (event prehistoric ones) and all event fields in memory all the time 
Yes, this could be avoided by indexing all the events based on time in cache and loading just the ones which the view needs. It would good to have this fixed in ECalBackendCache and make the file/exchange backend’s use the same.
To provide you with some data, my co-worker’s calendar is 1.1Mb and has 8562 events. He likes to keep old meetings around for reference.
Nice to see you are arriving to the same ideas I had after your first post
http://slusnys.wordpress.com/2008/05/22/eds-and-its-cache-analyze-this/#comments
Please contact me to my private e-mail address if you’d like to discuss my previous comments. I cannot find yours and when I looked on IRC for the user slusnys, it hadn’t been online for like 40 days, so that’s why I haven’t contacted you directly, yet. In case you don’t have access to my e-mail address, the username is jkohen and the domain is users period sourceforge period net. I’d be happy to discuss my previous posts!
I’ve experienced similar performance suckage with libical at large numbers of events.
Similar problem to mail boxes.
It’s a problem of indexes and the manner in which the data is stored.