You really got me interested in the numbers of free... so I googled to get more information about free and found the following information:
Memory Management.
So basically the "buffers" and "cached" column can be ignored ("shared" should always be zero). So the only interesting column is the "-/+ buffers/cache" line.
I did some free statements on my 64 Bit 10.04 Desktop:
1) System started, console login, and "sudo stop gdm"
CODE
total used free shared buffers cached
Mem: 4056800 954556 3102244 0 205096 478052
-/+ buffers/cache: 271408 3785392
Swap: 11885560 0 11885560
2) Starting gdm again (still using console)
CODE
total used free shared buffers cached
Mem: 4056800 1029928 3026872 0 205064 479136
-/+ buffers/cache: 345728 3711072
Swap: 11885560 0 11885560
3) Logging out and switching to the graphical log; after login just starting a terminal. I have many compiz effects enabled.
CODE
total used free shared buffers cached
Mem: 4056800 1216984 2839816 0 205172 483268
-/+ buffers/cache: 528544 3528256
Swap: 11885560 0 11885560
4) Used another User, without any extreme UI effects, basically default-fresh Ubuntu login
CODE
total used free shared buffers cached
Mem: 4056800 1149540 2907260 0 204824 482128
-/+ buffers/cache: 462588 3594212
Swap: 11885560 0 11885560
So my system uses between 271408 and 528544 kilobytes ... that's about ~265 MB to ~516 MB. IMHO not too bad. I don't have a pre 10.04 system around, so I can't compare those numbers.