Topic 'Re: New 8 version of AntiVir using way more ram?' in forum 'Security' - dslreports.com http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20353830 en Fri, 25 Mar 2022 10:45:33 EDT Fri, 25 Mar 2022 10:45:33 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20383588
"It is simply wrong to claim it is not accurate. It accurately reports the current assignment."

Sure but users expect it to be "real time" and well as you agree .. it really isn't .. so that is a windows "quirk" is all ..

"Task Manager does reflect reality."
.. only for the "Windows concept" of WS .. not real mem used .. and this is what confuses users and creates problems for devs.. us devs dont "assume" .. users do however .. what is the "official explaination" for that one? So yeah we get reduced to calling explicit API calls *if* we know better? .. right well the purpose of my post is to clarify that for all devs and users..

Great to hear Balmer is bringing back XP :)

Common sense rules all..

"If you minimize the app, then the memory occupancy is reduced, and Task Manager will reflect that. It's not a case of it updating its figures, it's a case of the OS paging out parts of the VM that were previously resident."

OK TOTALLY WRONG as any dev knows.. The reality is THIS .. ONLY WHEN an app "happens" to be minimized .. would Task Manager be kind enough to reflect reality.. sry that is a flaw bro.. not a "feature" .. no matter what way you look at it.. user or dev.. its a FLAW.. an OS flaw or Task Man flaw? you can decide .. but to not reflect reality for so damn long and then suddenly decide on API call or min that it would update = NOT GOOD ENOUGH.. not in a server that may take 10000 connections and 2gb+ mem.. When was the LAST TIME .. you "minimized" a windows Service .. that umm should have NO WINDOW .. ok .. maybe you don't code? like WTF with you? *IF* you did code? You are as upset as the rest of us.. no? So yeah .. your "windows world" is not working in reality.. or do you call specific API and accept that your owned cos Windows is your only option? like WTF? Any decent dev rebels against that and wants it FIXED..

Unix is MUCH better for that ..

Free should be enough to reflect reality.. just like other OS.. "in reasonable time" .. just no .. NOT "windows" .. you cant justify that to me man .. sry

If M$ wants to compete and convince both devs and users? Its time to stop throwing chairs, admit Vista is an abject failure and well reflect what other SERVER OS can do .. lame excuses that are Windows specific do NOT wash with me or others..

Btw Dave? If I was ever to throw a chair .. it would NOT be in your direction :)

However that statement .. does reflect reality if your a neutral in all this.. accept it as a flaw and well why should we have to? Expect better .. and maybe "better" comes..

I am one .. but I am Legion .. 100 more come after me .. asking same questions .. see?]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20383588 Fri, 25 Apr 2008 01:06:14 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20372844 is the number of page frames assigned to the process.

It is simply wrong to claim it is not accurate. It accurately reports the current assignment.

... when an app frees that 40 mb ...
An app that 'frees' 40MB in fact typically returns that virtual memory to an internal process heap. The VM is still in use by the same process as far as the OS is concerned. Nothing has happened except some internal user-mode accounting (within that process) of what the VM is used for. This does not change the working set size of the process and has no effect on real-memory occupancy. Thus, Task Manager reports no change, because there is no change.

As a general rule, this is good and sensible OS design. That's why the same sort of scheme is used in most general-purpose operating systems. You don't want to go making transitions into the kernel every time you want to allocate or free some dynamic memory; the overhead is tremendous.

There are 3 different mechanisms at work here:

1. The process's internal organization, which is its business alone. It allocates memory from and frees memory to its own heap. This is not visible in any Task Manager column, because the kernel doesn't know about this, because the kernel is not intended to know about this.

2. Allocation of virtual memory by the OS to the heap, when necessary (see 'VM size' in Task Manager). By 'when necessary' I mean, for example, that the original request for 40 MB probably won't be satisfiable from the heap, so the heap gets expanded.

3. Allocation of real memory by the OS to hold the virtual memory of the process (see 'WS size' in Vista's Task Manager; offhand, I can't remember whether 'Mem' in XP shows the WS size or the private commit for the process).

In short, a process can sit there doing:

allocate 40 MB
free 40 MB

all day long without changing the numbers reported by Task Manager, because the process size DOES NOT CHANGE after the first allocation, and because there's no inherent reason for the OS to trim its working set either.

Exactly where is the "confusion" in this statement?
"Task Manager Lies and is REAL slow to reflect reality.. "
Task Manager does reflect reality. If it says the WS size is 1234 MB, then the WS size really is 1234 MB.

The point of the display is to allow you to gauge how much memory is being used, and there's your answer: 1234 MB. The internal gyrations of the process ("am I actually using this?") do not affect this number.

If you minimize the app, then the memory occupancy is reduced, and Task Manager will reflect that. It's not a case of it updating its figures, it's a case of the OS paging out parts of the VM that were previously resident. The answer to 'how much memory is being used' is now less than it was before. The internal gyrations of the process are not affected by this change.

Task Manager has been reflecting reality all along. The problem is that you're making assumptions about how the process should behave, but it doesn't behave in the manner you assumed. 'Freeing 40 MB' is an internal action of the process with no external effect.]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20372844 Wed, 23 Apr 2008 08:04:38 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20372297
That is exactly my point .. users EXPECT that the numbers in Task Manager are accurate.. its like ASSUMED .. and no amount of Tech BS would chg that .. But your right .. the Info you saw was the same info as Task Manager .. and well OS Flaw.. aka Free does not work .. ah well once we know etc..

Thx for thumbs up btw .. I hope I helped bring a clearer understanding .. It would be the only reason I posted in your topic :)]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20372297 Wed, 23 Apr 2008 01:22:35 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20372084
We basically agree but there is no "confusion" on my part:

1) A program that knows it has consumed a large working set but is now going to be using significantly less, can voluntarily purge its own working set.

"This is good citizenship in a shared system"..

Actually, it isn't .. Its an expensive API call (to be avoided) and only to be called in very specific circumstances.. but reading between the lines, I think you understand that.

In Original Post, the excessive mem usage that occurred after an update and probably disappeared after a reboot.. a specific call on the API was probably appropriate in that case and the Task Manager did NOT reflect the true situation at all :)

2) the Task Manager is seen by users ( a common perception which is FALSE) as "real time" and "accurate" .. and well it isn't at all .. when an app frees that 40 mb? It wont reflect that state in "Task Manager" until a "minimise" or an API call.. or as you state .. at some random time it chooses which does not happen quickly.

Firefox could call the API in certain cases (it currently does not).. but .. how exactly would it know when it needed to (maybe watch for closed tabs?) .. However, I think there are still some other mem issues in FF that need to be worked out by more traditional means. Nothing is perfect.

The real problem is that Windows does not update often enough the "WS size for process X" .. rather that is left to the app dev which is a little "funny" when all he should really have to do is call "free"? Anyway .. that is life .. :)

But yeah .. task manager "lies" as far as I am concerned .. maybe that is because users dont understand how it works or whatever .. but it is no real time "monitor" .. I would be better off using "Space Invaders" to determine what phys mem was in use :)

PS:
"2) Task manager may or may not periodically update its mem column (I don't know) but that's not a consequence of whether apps purge their working sets explicitly. Task manager is just a display tool."

Who DOES know? I want to speak to them .. also ... it seems to be absolutely explicit .. if you don't MANUALLY purge it.. your reported for high mem usage .. I am not sure that is "fair" .. are you? Most "users" see the Task Manager data as "gospel" .. I consider that a HUGE OS flaw.. One to add to the many that precede it ..

Exactly where is the "confusion" in this statement?
"Task Manager Lies and is REAL slow to reflect reality.. "

Free is not enough? WTF etc? ]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20372084 Wed, 23 Apr 2008 00:15:24 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20360675 --
Only those defenses are good, certain and durable, which depend on yourself alone and your own ability. -The Prince, by Niccolo Machiavelli.]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20360675 Sun, 20 Apr 2008 22:31:11 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20359212 --
Never argue with an idiot. They just bring you down to their level and beat you with experience.]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20359212 Sun, 20 Apr 2008 16:13:19 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20358910 http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20358910 Sun, 20 Apr 2008 14:56:01 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20358195
But don't worry about it; the system can reclaim memory when needed. That's probably what happened between you observing the WS at 45MB and observing it at 8MB.

The large occupancy probably happened after scanning something 'large', I would guess.

As long as the WS doesn't slowly climb without limit, you're ok.]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20358195 Sun, 20 Apr 2008 11:19:25 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20358186 said by m0d:

"The working set of the specified process can be emptied by specifying the value (SIZE_T)–1 for both the minimum and maximum working set sizes. This removes as many pages as possible from the working set. The EmptyWorkingSet function can also be used for this purpose."

So yes .. any app hogging 30-40 mb should be calling this .. just so Task Manager knows the "score".. otherwise it will continue to "lie".. You're confusing two different things.

1) A program that knows it has consumed a large working set but is now going to be using significantly less, can voluntarily purge its own working set. This is good citizenship in a shared system, but of course one need to be sure that purging doesn't cause more overhead than not purging (purging pages you actually need means they need to be immediately paged back in). Note that a program that does not explicitly purge may still be subject to automatic working-set trimming.

2) Task manager may or may not periodically update its mem column (I don't know) but that's not a consequence of whether apps purge their working sets explicitly. Task manager is just a display tool.

If you start a new instance of Task Manager and it says the WS size for process X is 'n MB', then the WS size for process X is 'n MB'. Task Manager is not lying at all; that number reflects the actual WS assignment for the process.

===
The observed behaviour is simply how paging works (which is how it should work):

An app can voluntarily give up its working set if it knows it is 'going idle' (this is typically something background programs ought to worry about, but not interactive apps).

An app can have its working set trimmed if the OS has other uses for the memory. This is the 'normal' way that processs have their memory occupancy reduced.

There is no point at all in the OS taking away working set from any app unless there is immediate need for that memory. Memory sitting around unused is wasted.

For example .. start some app taking 40+ mb .. MINIMISE it .. now look at it after minimise in Task manager. Apparently only on minimise does Task Manager update its view of mem usage.
That's perfectly explained by the process working set being purged on minimize.

Probably Firefox does it for itself, though the shell could also be doing it (it makes some sort of sense to purge the WS on minimize, since we won't be needing a lot of the virtual address space -- like the parts containing the UI -- right now).

The shrinkage doesn't seem to happen with IE, so maybe it's Firefox itself doing it.

===
Postscript: it occurred to me that some of the confusion here may be due to failure to appreciate

(1) the difference between the real-memory view of a process (its working set) and the memory that the process itelf sees (which is purely virtual, and may or may not be represented in the working set).

(2) how memory seen as 'free' by application code is in fact still in use as far as the OS is concerned; in particular, use of process-local heap storage in order to make dynamic storage allocation affordable.]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20358186 Sun, 20 Apr 2008 11:08:14 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20357940 http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20357940 Sun, 20 Apr 2008 09:47:50 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20357151
For example .. start some app taking 40+ mb .. MINIMISE it .. now look at it after minimise in Task manager. Apparently only on minimise does Task Manager update its view of mem usage. Its totally possible an update took 40 mb for a few secs.. but Windows Task Manager is probably continuing to tell lies about mem usage a few mins later :)

Ah well .. once we know ..

Edit:

See this function in WinAPI:
»msdn2.microsoft.com/en-u ··· 234.aspx

"The working set of the specified process can be emptied by specifying the value (SIZE_T)–1 for both the minimum and maximum working set sizes. This removes as many pages as possible from the working set. The EmptyWorkingSet function can also be used for this purpose."

So yes .. any app hogging 30-40 mb should be calling this .. just so Task Manager knows the "score".. otherwise it will continue to "lie"..

EDIT2:
Anyone know if this if fixed in "vista" because its an AGE OLD problem for devs?

Edit3:
And Process Explorer .. is only reporting the Task Manager view of things which is distorted.. call it an OS Flaw .. a MAJOR one..

I have seen so many devs try to explain this to users and well the dev is ALWAYS the LOSER.. all cos Task Manager lies really bad.. hmm

Edit4:
Start Firefox a good example of this (that said it does leak handles) .. look in Task Manager.. 20 mb ++ ... Minimise it .. look in Task Manager again .. its 2mb .. what should that tell you? Task Manager Lies and is REAL slow to reflect reality.. funny how some trust it though and will argue ad nauseum.. ah well ..

To Orig Post guy (Sentinel) .. I think you have nothing to worry about. Its "normal" in an XP "Task Manager" .. etc .. just a blip :)]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20357151 Sun, 20 Apr 2008 00:21:35 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20355331 I think it also varies around a bit depending on what it's doing at the moment. ]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20355331 Sat, 19 Apr 2008 16:29:32 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20355165
I swear for 2 days it said it was using 45mb and now it is back t normal :( Oh well. I'll keep an eye on it.

I have no auto stuff scheduled at all and I don't use auto update so I don't know what it could have been.]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20355165 Sat, 19 Apr 2008 15:51:37 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20354464 http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20354464 Sat, 19 Apr 2008 12:37:09 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20354328 http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20354328 Sat, 19 Apr 2008 12:01:31 EDT Re: New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20353922
Click for full size
]]> http://www.dslreports.com/forum/Re-New-8-version-of-AntiVir-using-way-more-ram-20353922 Sat, 19 Apr 2008 10:07:45 EDT New 8 version of AntiVir using way more ram? http://www.dslreports.com/forum/New-8-version-of-AntiVir-using-way-more-ram-20353830 http://www.dslreports.com/forum/New-8-version-of-AntiVir-using-way-more-ram-20353830 Sat, 19 Apr 2008 09:37:15 EDT