The crawl is running far slower than it should – SharePoint Search

SharePoint Server 2007 – 10 suggestions for a reliable, low-latency Search deployment see article here from agazzeri’s Blog

My customer built a 64 bit index server to take on the load of MOSS Search.  The server has 72GB of
memory, but only uses about 10GB at max.  The crawl is running far slower than it should be.  Does
anyone have any idea why the memory use might be so low?

Summary

Check the registry settings around the dedicated filter process quota.  Default in MOSS, the search
process only uses about 100Mb.

Fine tune the Gatherer. The three registry keys below (under  HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office
Server\12.0\Search\Global\Gathering Manager) can be changed for better  performance (this will require a service restart for the changes to take effect)

a.  FolderHighPriority: Represents the number of high priority folders  that can be processed at one time.  If this is too high then the cache in  the daemons will constantly be running out of space.  If this is too low
then the crawl will be throttled waiting for more items to process.

b.    FilterProcessMemoryQuota: Represents how much memory can be consumed by the search daemon process before it gets killed by the crawler. The OOB default has been chosen based on 4 GB of memory on the indexer. If the customer has higher RAM, they can increase this value to cache more data during the crawl.

c.     DedicatedFilterProcessMemoryQuota: Same as for FilterProcessMemoryQuota except this is the size of the single-threaded daemons. As an example, if your indexer box is 64-bit with 16 GB of RAM, the following values have been tested successfully:

a.       FolderHighPriority: set to 500
b.      FilterProcessMemoryQuota: set to 208857600
c.       DedicatedFilterProcessMemoryQuota: set to 208857600

Advertisements