Introduction

the following document outlines the process and values for tuning the folder scan process for scanning large filesystems the system defaults should be valid in most situations, however, if the folder scan is taking too long then the following may help debug the issue or tune the system to speed up the process a typical scan of a remote file system should scan at least 3000 4000 folders per minute slower than this can indicate a network or file system issue note small scans or shares can fall outside this range trace logging all folder scans and output are logged to %activate%/log files/activate filesync log the default settings log the start and stop of all actions and some statistics on the length of time different actions have taken the trace level may be adjusted for debug purposes by setting a 'trace' parameter to a value between 0 (default) and 4 (max) it is not recommended to leave the trace level at any value other than 0 in a production environment unless instructed by an activate support engineer scan jobs the system iterates though each item under //resources/folders (and children) and adds each top level unc share as a 'scanjob' so the scan jobs are effectively added in alphabetical order note these 'scanjob's are internal and are not activate jobs concurrent scan jobs the system will create a thread pool to run a number of scanjobs in parallel the number of concurrent threads (and therefore scanjobs) defaults to 4 processors in the system this can be altered by setting a 'maxthreads' parameter at the root of //resources/folders this value may need to be reduced if you have a large number of processors in your system and your sql environment cannot work with the large number of threads scan job order the order that the scanjobs are run in can be adjusted by setting a parameter 'scanpriority' a higher scan priority will result in the scan of that folder being scanned before a folder with a lower priority the default priority is 0 frequency by default a unc share is scanned every time the overnight scan process is run however, this can be changed by using a 'frequency' parameter for a specific unc share to control how often it is scanned these values work in conjuction with the 'lastscan' parameter options are 'never' this will disable the scan this is the same as unpublishing the folder node in activate +xdays this will only scan the folder every x number of days since the last scan days of the week (ie monday, tuesday, etc) the folder will only be scanned if the current day is one of the specified days scanning large shares in parallel by default, each unc share is scanned in a single scanjob however, some large shares or dfs roots will benefit from splitting the sub folders into seperate scanjobs this is done by setting a 'scanparallel' parameter as the root of the unc share that should be split into seperate parallel scanjobs each sub folder will then be scanned in parallel warning please ensure that your file server can handle multiple scans scanning in parallel may infact slow down the scan if the file server cannot cope the file system should be scanned one without the parameter on to determine a base line the 'scanparallel' parameter can then be set and a new scan run to determine if it may the scan faster or slower excluding folders it may be beneficial to exclude some types of the folders and data from the scan for example, system import or system specific folders can automatically be exclueded and not scanned by the system this can also improve the scan times this can be set by providing a excludefolders parameter as a comma seperated list of folder names to exclude the folder names can use a ' ' as a wildcard at the beginning and/or end of the name for example, 'system ' would exclude all folders starting with 'system ' folder scan depth the depth of the folder scan should be kept as low as possible large depths (>7) can result in large numbers of folders being scanned and long scan times a good design for a file system should keep secured folders at levels 3 or 4 and higher securing folders at lower depths makes it very difficult for users to find and navigate this information and increases issues with list access to these folders autounpublishtrusteeusagecount groups can sometimes be misconfigured to grant access to multiple folders activate will warn users can approvers when this happens but allow the action if approved it is possible to detect this case an automatically disable these groups in activate this function is a global and should be used carefully and in a uat environment first to determine it's effect in a specific environment the default value of 0 means this function is not enabled any value greater than 0 is enabled if a group is scanned and it is used in more that n places when it will be unpublised as an inappropriate group for folder access a typical value of 3 4 is a balance between disabling lots of groups and security summary the following is a summary of the available parameters to control scanning of your folders true left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type