Skip to main content
Policy elements are building blocks you can use when you create WebMarshal policy groups and rules. These elements help you to specify complex rule conditions and rule actions. Some examples of each type of element are provided by default when you install WebMarshal with the default configuration. These examples are used in the default access policy. You can edit the existing elements or create new ones to support your policy requirements.
You can see how WebMarshal uses any item by viewing the properties of the item. For example, a User could be included in User Groups and used in Rules. A URL Category could be included in other Categories and used in Rules.
  • To view the properties of an item, select it and click the Properties icon in the toolbar, or use the right-click menu.
  • To open a rule or category that uses an item, select it in the properties window and then click Jump To.
  • For details of the available options, see Help for each Properties window.
WebMarshal provides the following types of elements: User Groups Allow you to apply policy based on user accounts or workstation IP addresses. For more information, see User management. URL Categories Allow you to apply policy based on requested URLs. Categories can include manually entered URLs as well as information obtained from external filtering lists. For more information, see Understanding URL categories. URL Filtering Lists Allow you to configure external lists sources for URL categories, including text based imports, DNS Blacklists, and the LevelBlue Web Filter Database. For more information, see Configuring URL filtering lists. Schedules Allow you to apply policy based on the time of day and day of the week. For more information, see Configuring access using schedules. TextCensor Scripts Allow you to apply policy based on the text content of uploads and downloads. For more information, see Identifying web content using TextCensor scripts. Classifications Allow you to classify files and requests based on rule criteria. For more information, see Logging activity with classifications. Quotas Allow you to limit browsing activity by time and bandwidth. For more information, see Configuring access using quotas. Malware Protection Allows you to check downloads and uploads for viruses, malware, and other malicious content using third party scanners. For more information, see Scanning overview.

​
User management

The User Management functions allow you to import and organize user account information. This information is used to control and record browsing access.
Connectors, previously covered in detail here, are now covered under Global Settings. See Configuring connectors. You can choose to create a new connector any time you import a User Group.

​
User groups

​
All users

Select this element to view a complete list of user names in all groups that have been imported into WebMarshal. This list is empty until at least one group has been imported (or, in the case of IP range entries, a computer in the range has browsed through WebMarshal).
By default WebMarshal retains all previously imported user names on this list, even if they are not currently members of any user group visible within WebMarshal. To purge unused names, right-click All Users and select Purge Unreferenced Users. This action does not affect logging records.

​
User properties

To view additional details about a user, double-click to open the User Properties window.
  • The General tab shows the name, description, source, and full distinguished name information for a user. You can edit the description.
  • The Rules tab shows all rules that apply to this user. Highlight a rule and click Jump to Rule to view the details of that rule.
  • The User Groups tab shows all user groups that this user belongs to. Highlight a group and click Jump to User Group to view the details of that group.
  • The Quotas tab shows a list of quotas that apply to the user. To extend a quota for the user, click Extend Quotas. For more information about extending quotas, see Editing a quota.

​
User group types

WebMarshal supports five types of User Groups. Each displays in the Console with a unique icon.
  • WebMarshal groups (user folder).
  • Windows NT user groups (green shirts) .
    WebMarshal supports both Global and Local NT user groups.
    • Global user groups are created on a Windows Domain Controller or Active Directory server.
    • Local user groups are created on domain controllers or standard workstations, and can contain users from any domain. They can also contain global user groups. Local user groups cannot contain other local groups.
  • Active Directory user groups (blue shirts).
  • Novell NDS user groups (red shirts).
  • IP address range groups (users with workstation).
Typically, WebMarshal groups include users that have similar Web access requirements. WebMarshal groups can include other groups, single users, and computer groups. The default configuration that you can import when you install WebMarshal includes several user groups. To quickly enable the default policy, you can import groups from connectors, or create computer groups, and then insert these new groups into the default WebMarshal groups.
WebMarshal Console showing the User Groups list with Import User Group, New IP Address Group, New WebMarshal Group, Insert Existing, and Delete toolbar actions

WebMarshal Console, User Groups window

To view the list of users and groups contained in a group, select that group in the left pane menu tree. To edit the group description, and/or to reload the group (for Windows and NDS directory groups), right-click the group and click Properties. For network security, groups imported through connectors are read-only and cannot be edited using the WebMarshal Console. These groups are synchronized with the parent directory on the schedule you specify in the Connector properties for the appropriate directory type (by default, once a day).

​
Adding a user group

  1. In the WebMarshal Console, expand User Groups or a specific group.
  2. From the Action menu or the right pane menu bar, choose one of the following actions:
    • Import User Group
    • New IP Address Group
    • New WebMarshal Group
  3. Import User Group(s) from a Connector: On the Import User Groups window, find and select the names of the groups you want to import into WebMarshal. You can use imported groups, and individual users, in WebMarshal user groups or rules. You can explore or search for groups. You can also enter group names manually. For NT groups, names you type must be in domain\usergroup format. Enter multiple names separated by semi-colons. For Active Directory and NDS, names you type must be fully distinguished names.
    WebMarshal can import groups from trusted Active Directory domains, subdomains, and other domains that have an explicit two way trust relationship with the domain that WebMarshal is a member of. For additional details see LevelBlue Knowledge Base article Q11870.
    Import User Groups window for the Windows NT connector with Explore, Search, and Manual tabs

    Import User Groups window

  4. Click Import to add the User Group.
  5. New IP Address Range Group: Enter a name and optionally a description. Enter an IP address, a range (starting and ending addresses separated by -) or a CIDR specification. Any user or process on a computer with an IP address in the specified range can browse through WebMarshal, subject to any Rules applied to the group.
    A range can be in the IPv4 or IPv6 address space.WebMarshal adds the name or IP address of each computer in this range to the All Users list the first time that computer connects to WebMarshal. Once the computer names are included in the list, you can use them individually within Rules in the same way as any other User.If you want to add a specific computer to a WebMarshal User Group explicitly (when it is not included in All Users), see Adding computers or IP addresses to a group.
    IP authentication works much better if DNS is configured to support reverse DNS lookups. Reverse DNS allows WebMarshal to get a name for an IP address. If that returned name matches the NetBIOS name, WebMarshal can query the computer for its description. (As a consequence, you cannot specify different permissions for the IPv4 and IPv6 interfaces of the same computer.)
    New IP Address User Group window with name, description, and an IP address range entered in CIDR format

    New IP Address User Group window

  6. New WebMarshal Group: Enter a name and optionally a description. This type of group is internal to WebMarshal. You can use WebMarshal groups to quickly apply policy to multiple groups and users.

​
Inserting existing groups to a WebMarshal group

  1. In the WebMarshal Console, expand a specific group within User Groups.
  2. Click the Insert Existing icon in the tool bar. If no users or imported groups are present, you have the option to import or create User Groups (see the procedure above)
  3. Within the Insert Users or Groups window, select Existing Imported Users or WebMarshal Groups. Select one or more users or groups with the mouse, or type the beginning of a name to select it.
    Insert Users or Groups into Power Users window with Existing Imported Users or WebMarshal Groups selected and a filterable list of users and groups

    Insert Users or Groups window

  4. Use the Enter key or click Add to add the selected items into the Group.
WebMarshal also supports “drag and drop” for inclusion of members in a group. To use this feature, drag a group or user name or names (from either pane) over a group name in the left pane. Hold down the Ctrl key while dragging to copy the group or user name; otherwise it will be moved.

​
Adding computers or IP addresses to a group

  1. In the WebMarshal Console, expand a specific group within User Groups.
  2. Click the Insert Existing icon in the tool bar.
  3. Select New Computer Accounts by IP Address or New Computer Accounts by Computer Name. In the text box, enter computer names, or IP addresses, one per line. You can only add valid computer names from the local network. IP addresses are resolved to computer names if possible. You can add any well-formed IPv4 or IPv6 address.
    WebMarshal attempts to resolve IP addresses to NetBIOS names. NetBIOS names are used to allow for dynamic allocation of IP addresses. You can enter fully-qualified domain names, but they might be rejected due to IP address duplication.

​
Changing user group properties

The User Group Properties window shows the source and reload status of a User Group. Use this window to:
  • Edit the group description for WebMarshal and IP groups.
  • Change the range of IP addresses included in an IP address group.
You can also click Reload Now to update the group membership, provided the group has an external source (Windows NT or NDS directory). Reload also updates the Status information with the result of the reload operation. To schedule automatic reloading of the group, edit the Connector properties.
Reloading membership does not delete any users previously imported, even if they are no longer members of the group. To remove unused users, select User Groups > All Users > Purge Unreferenced Users.

​
Understanding URL categories

URL categories are collections of related Web or FTP sites. You can use these lists in Rules to determine an action to take (such as permitting or blocking access, or warning the user).

​
Types of URL categories

WebMarshal supports two types of URL Categories.
  • WebMarshal URL Categories are user-modifiable lists of URLs. This type of category may contain other WebMarshal Categories and may also contain imported Categories.
  • Imported categories are populated with URL information supplied by an externally maintained Filtering List (for instance the LevelBlue Web Filter Database or WebMarshal’s FileFilter). You cannot edit the contents of an imported category within WebMarshal.
You can use either type of Category in rule conditions. WebMarshal creates several default Categories during initial configuration. You can also add or import categories at any time.

​
WebMarshal URL categories

Several categories are provided by default. Some examples of WebMarshal URL Categories are:
  • Offensive Content
  • Business Partners
  • News Sources
  • Web-based Mail Servers
  • Search Engines
You can use text files to export or import URLs. WebMarshal TextCensor Rules can also add URLs to categories. WebMarshal allows URLs to include an entire site, a particular sub-directory (path) at a site, or a specific path and file name. The * wildcard character can be placed at the beginning or the end of the site name part of the URL to match multiple similar sites.
Matching of URLs is not case sensitive.The path, file, and query string parts of a HTTPS URL (for instance https://example.com/folder/) can only be evaluated if HTTPS content inspection is enabled. WebMarshal does not have access to the path information for uninspected HTTPS connections.
To see a list of the URLs and Categories included in a Category, expand it in the Console tree. To view more information about a category, including a list of the Rules and other Categories that use it, view the Properties of the Category. For details of the available information, see Help. If a URL was placed in a Category by WebMarshal Rule action, the URL listing includes the date and time when it was appended. To see a report of the WebMarshal Rule action that added a URL, double-click it. (If the URL was added manually, no information is available and double-clicking will have no effect.)
WebMarshal Console showing the URL Categories list with URL, child category, and filtering list category counts

WebMarshal Console, URL Categories window

​
Adding a URL category

  1. Select URL Categories in the left pane.
  2. Click the New Category icon in the tool bar.
  3. Enter the name for the new category. Optionally enter a description.
  4. On the final page of the Wizard, click Finish to create the Category.

​
Adding URLs to a URL category

  1. Select URL Categories in the left pane.
  2. Select the name of a URL category in the left pane to view its members in the right pane.
  3. Click the Add URLs icon in the tool bar to open the Insert URLs window.
  4. To add a new URL to this category, enter the URL. You can enter any of the following:
    • A full site name or IP address literal, and optionally a port, such as:
      • http://www.levelblue.com/
      • http://198.51.100.25/
      • http://[2001:db8:8:4::6]:81
    • A specific sub-directory, such as http://support.levelblue.com/webmarshal/
    • A specific full path including file name, such as http://support.levelblue.com/webmarshal/documentation.asp
    • A string with an asterisk as a wildcard at the beginning and/or the end of the site part of the URL, such as http://*.wave* (not available with IPv6 literals)
    • Any of the above, with a query string. For details of query string matching rules, see URL query string matching.
  5. To clarify that the entry includes a file name, check the box URL includes a file name. This box is disabled when the last character of the URL entered is / (slash). (URLs with a file name do not have a slash at the end of the resulting URL.)
  6. Optionally select the variations of the URL that you want to add (such as the www. prefix or HTTPS and HTTP protocols).
    Insert URLs into Advertising window with a wildcard URL, URL includes a file name checked, variation options, and a comment

    Insert URLs window

  7. Click Add (or press Enter) to add the URL to this category.
  8. The window remains open so you can add additional URLs.
  9. Click Close when done.

​
URL query string matching

When you manually add a URL to a URL Category, you can include a query string part. WebMarshal will match URLs that contain the query string text you enter. Like all URL matching, query string matching is not case sensitive. You cannot use wildcards in the query string part.
  • Begin the query string part of the URL with ?
    • For example, http://example.com/?query=1
  • The query string can contain one or two strings to match exactly. Separate the strings using &
    • For example, http://example.com/path/file.htm?this=1&that=text
  • If you include two strings, both must be present for the URL to be matched.
    • http://example.com/?this&that does not match http://example.com/?this
  • The strings will be matched in any order, and will match a URL with additional query parts.
    • http://example.co*/?this&that matches http://example.com/?that&more&this
  • If you add query strings to an entry, they match on any deeper path.
    • http://*.com/?this&that matches http://example.com/path/file.htm?that&this&more
  • Strings must match exactly.
    • http://example.com/?attach does not match http://example.com/?attached
  • To match a key-value pair you must enter the exact key=value string.
    • http://example.com/?true does not match http://example.com/?this=true
Characters in the query string should be urlencoded. For example, a literal ampersand (&) in a string should appear as %26 (but the & used as a separator for parameters must not be encoded).To be sure of matching spaces you may need to match both + and %20 (for example, http://example.com/?a+space&a%20space).

​
Adding categories to a URL category

  1. Select URL Categories in the left pane.
  2. Select the name of a URL category in the left pane to view its members in the right pane.
  3. Click the Insert Categories icon in the tool bar to open the Insert Categories window. The Insert Categories window shows all available categories in a tree view, organized by the source (including WebMarshal categories and filtering list categories).
    Insert Categories into All URL Categories window showing a tree of WebMarshal and FileFilter categories

    Insert Categories window

  4. To include one or more URL categories in this category, expand the tree and select the desired categories.
  5. Click OK to add the selected categories and return to the parent window.
You can also “drag and drop” a category name or names (from either pane) over a category name in the left pane. If you want to copy the category, hold down the Ctrl key while dragging.

​
Importing a URL category

You can import URLs from a file created by another application or WebMarshal installation. The imported URLs can replace or add to the existing content of the category. The file must be a simple text file with one URL per line.
You can also import categories from files using the WebMarshal FileFilter filtering list. With FileFilter, you can maintain the category files outside WebMarshal, and import them daily. FileFilter is also a better choice for categories containing large numbers of URLs. See FileFilter.
To import URLs to a Category:
  1. In the Console menu tree, expand URL Categories and select the category you want to use.
  2. Click the Import URLs icon in the toolbar.
For more information about results and error handling, see Help.

​
Exporting a URL category

You can save the members of a URL category to a file that you can export to another application or WebMarshal installation. To export a URL category:
  1. In the Console menu tree, expand URL Categories and select the category you want to use.
  2. Click the Export to File icon in the toolbar
  3. Select the location where you want to save the category file. Click Save.

​
Searching a category for a URL

To search a Category for a specific URL:
  1. Select a WebMarshal category in the tree, then click the Find URL icon in the toolbar.
  2. Enter the name of the URL to search for.
  3. If you want to search all categories, select Search all URL categories.
  4. Click Find.
  5. The result indicates whether the URL is in the currently selected category. If you selected Search all URL categories, the result indicates which categories if any the URL is in.

​
Configuring URL filtering lists

Lists are categorized lists of websites, maintained externally to WebMarshal. You can use the categories provided by these lists in WebMarshal URL Categories. The Filtering Lists pane displays information about the lists currently configured in WebMarshal. You can enable or disable an existing list. You can add or delete lists. WebMarshal currently supports the following lists:
  • FileFilter: A simple implementation that reads category lists from text files. For more details about FileFilter please see FileFilter.
  • URLCensor: A real-time facility that categorizes IP addresses through DNS Blacklist lookup. For more details about URLCensor please see URLCensor.
  • LevelBlue Web Filter Database: A URL list provided by LevelBlue and licensed separately. The list consists of 116 categories. For more information on the LevelBlue Web Filter Database including trial license information, please see LevelBlue Web Filter Database.

​
Reviewing filtering list status

You can check the available categories, enabled or disabled status, and update status for any configured list, and update the list or license if required. To check the list:
  1. In the left pane tree, expand URL Filtering Lists.
  2. Select a list. The top right pane shows the status of the list on each server. The bottom right pane shows the name and description of each category available through the list.
  3. To view general details of a list, or to initiate an update for all servers, right-click a list name in the left pane. Click Update All Servers to force an immediate check for Filtering List updates (where internet updates are provided).
    Updating can cause disruption for users browsing. This action should be performed at a time when it will have minimum effect.
  4. To view details of list and license status on each server, double-click a server name in the right pane. The Node Update Status tab shows the update status and results for the server.

​
Adding filtering lists

You can add one or more filtering lists to the WebMarshal configuration. For additional details, see Help. To add a filtering list:
  1. In the left pane tree, expand URL Filtering Lists.
  2. In the right pane, click Add URL Filtering List to start the Add URL Filtering List wizard.
  3. Select a single list to add. Click Next.
  4. For the LevelBlue Web Filter Database, to allow WebMarshal to validate the license (one time requirement), select an Internet connection method that is valid from the WebMarshal Array Manager server, and then click Next.
  5. Click Finish to validate the license (if required) and add the list.
  6. To add another list, repeat the above steps.
  7. Commit configuration changes to propagate the new list to the WebMarshal servers.
After adding a filtering list, you can use the categories from that list by inserting them into WebMarshal URL Categories. See Adding categories to a URL category.
WebMarshal will use the categories for filtering as soon as the list is available on processing servers. For the LevelBlue Web Filter Database, the initial population of categories can take half an hour or longer, depending on Web connection speed.

​
Deleting a filtering list

You can delete a filtering list from the WebMarshal configuration. Deleting a list removes any categories imported from the list from the WebMarshal URL Categories. Deleting a list does not remove database log records of categorized sites. To delete a filtering list:
  1. In the left pane tree, expand URL Filtering Lists.
  2. Right-click the list you want to delete, and select Delete.
  3. Commit configuration changes.

​
Enabling or disabling a filtering list

You can enable or disable use of a Filtering List. When a list is disabled, the list and imported categories will display in the Console, but the list will not be updated and it will not be used in rules. To enable or disable a filtering list:
  1. In the left pane tree, expand URL Filtering Lists.
  2. Right-click the list you want to change, and select Enable or Disable.
  3. Commit configuration changes.

​
Configuring access using schedules

The scheduling facility allows you to apply Rules at particular times. You can set different Quotas or Web access permissions by time of day or day of the week. For instance, an organization might block access to games sites during working hours, but allow access after hours.
Schedule times are calculated in local time on the processing server that handles a request.
You can apply different schedules for different user groups. To list the rules that use a schedule, view the properties of the schedule.

​
Adding a schedule

  1. Select Schedules in the left pane.
  2. Click the New Schedule icon in the tool bar to start the New Schedule wizard.
  3. On the Time Schedule page of the wizard, drag with the left mouse button to add to the blue “inside” area. Drag with the right mouse button to erase from the blue “inside” area.
    New Schedule Wizard Time Schedule grid with weekday business hours selected, a Reset to Default Schedule button, and Snap to set to Half hour

    New Schedule Wizard, Time Schedule window

  4. Click Set Default Schedule to reset the schedule to the default time block.
  5. Choose to “snap” the schedule times to the nearest whole, half or quarter hour using the menu.
  6. Click Next.
  7. On the Schedule Name page of the wizard, give the schedule a name.
  8. Optionally enter a description of the schedule.
  9. Click Next.
  10. On the Completing page of the wizard, click Finish to save the schedule, or Cancel to exit without saving.

​
Editing a schedule

  1. Select Schedules in the left pane.
  2. Double-click an existing Schedule name in the right pane to edit it.
  3. Drag with the left mouse button to add to the blue “inside” area. Drag with the right mouse button to erase from the blue “inside” area.
  4. Click Set Default Schedule to reset the schedule to the default time block.
  5. Choose to “snap” the schedule times to the nearest whole, half or quarter hour using the menu.
  6. On the General tab, edit the name and optional description of the schedule.
  7. Click OK to save the changes to the schedule, or Cancel to lose any changes.

​
Duplicating a schedule

To duplicate a schedule:
  1. Right-click the Schedule name in the right pane.
  2. Select Duplicate.

​
Deleting a schedule

To delete a Schedule:
  1. Right-click the Schedule name in the right pane.
  2. Select Delete.

​
Configuring access using quotas

Quotas allow you to limit users’ browsing activity by time or by volume (total bandwidth). You can set different Quotas for different user groups. More than one Quota can apply to a user. Different limits can apply at different times. For instance, the organization may wish to limit access to news sites during working hours but allow unlimited access after hours. Quotas are applied as Rule conditions. To list the rules that use a quota, and the quota usage for each user, view the properties of the quota.

​
Adding a quota

  1. Select Quotas in the left pane.
  2. Click the New Quota icon in the tool bar to start the New Quota wizard.
  3. On the Quota Interval page of the wizard, select the allocation interval. Click Next to continue.
    Quotas are reset at midnight (local time at the WebMarshal processing server) at the beginning of the first day of the quota period.
  4. On the Quota Type page of the wizard, choose whether this will be a time or volume (bandwidth) quota.
  5. For a time quota, choose the amount and period (minutes or hours).
  6. For a volume quota, choose the number and unit (Kilobytes, Megabytes, or Gigabytes). Choose whether to apply the quota to pages served from the WebMarshal Proxy Cache.
  7. Click Next to continue.
  8. On the Quota Exceeded page of the wizard, select the actions WebMarshal will take when a user (or workstation) has exceeded the allowed quota.
    1. Select a web page to display from the list. (To learn how to add items to this list, see Notifying users with notification pages.)
    2. To allow users to extend the quota, check the box and enter the required information:
      • Amount of time or volume to be added per extension.
      • Number of extensions allowed.
      • Web page to display.
    New Quota Wizard Quota Exceeded page showing the QuotaExceeded page and an extension of 20 minutes up to 2 times with the QuotaExtend page

    New Quota Wizard, Quota Exceeded window

  9. Click Next to continue.
  10. On the Quota Name page, enter identifying information.
    • Enter a name for this Quota (required).
    • Optionally enter a description indicating the intended use of the Quota.
  11. Click Next to continue.
  12. On the Completing page of the wizard, click Finish to save the Quota, or Cancel to exit without saving.

​
Editing a quota

You can change the global properties of a quota. You can also extend a user’s allocation for the current quota period.
  1. Select Quotas in the left pane.
  2. Double-click an existing Quota name in the right pane to edit it.
  3. On the General tab, edit the name and optional description of the Quota.
  4. On the Quota tab, edit the properties of the Quota.
    Quota properties for 10 MB per Week, with a weekly volume limit, the option to apply the quota to cached files, and extension settings

    Quota Properties window, Quota tab

  5. On the Rules tab, you can see a list of rules that apply to this quota. Highlight a rule and click Jump to Rule to view the details of that rule.
  6. On the Usage tab, you can see the quota usage for a selectable period for each user affected by the quota.
    This display does not reflect Quota amounts used in current Active Sessions. The usage information is updated at the end of a browsing session.
    Usage tab for the 2 Hours per Day quota listing user accounts with amount used and percentage used, and Extend Quota, Refresh, and Select Columns buttons

    Quota Properties window, Usage tab

  7. To extend the quota for a specific user, select that user from the list then click Extend Quota to open the Extend Quota window.
    Extend Quota window for a user showing the quota usage date, a quota amount of 2 hours, and an extension of 15 minutes

    Extend Quota window

    • The window shows the user’s default allocation. Enter the desired extension as a number of minutes, hours, KB, MB, or GB. Click OK to apply the change and return to the Quota Properties window.
      Changes made here only affect the selected quota period (day or week). Changes made here do not affect the user’s ability to extend the quota (as set in the Quota properties).
    • To refresh the display, click Refresh.
    You can allow a user to extend their own quota when they reach the limit. See Quota extensions.
  8. Click OK to save the changes to the Quota, or Cancel to lose any changes.

​
Duplicating a quota

To duplicate a Quota:
  1. Right-click its name in the right pane.
  2. Select Duplicate.

​
Deleting a quota

To delete a Quota:
  1. Right-click its name in the right pane.
  2. Select Delete.

​
Quota levels

If WebMarshal Quotas are in use, users can check their quota allocation and usage from the WebMarshal website. The URL of the quota page is http://webmarshal.home/Quotas This URL can only be accessed from inside the local network for which WebMarshal is the gateway. The browser must be configured to use WebMarshal as a proxy server. This page shows the user name of the authenticated session. Each quota for that user is displayed. The quota limit (including extensions used if any), amount remaining, and percentage remaining are shown.
WebMarshal home page in a browser showing the 2 Hours per Day quota with 1.9 hours (97%) available

WebMarshal home page, quota levels

​
Quota extensions

If a user makes a Web request (page view, upload, or download) which would exceed their remaining quota, they will be notified by a web page. If the quota is configured to allow user extensions, and the user has not yet used all allowed extensions, the notification web page will include a button to extend the quota. When the user clicks the button Request a quota extension, the extension is recorded and the request is processed. The WebMarshal Administrator can monitor and extend quotas for all users from the Usage tab of the Quota Properties window (see Editing a quota).

​
Quota and browsing time calculation

WebMarshal calculates user browsing time for Quotas and to log usage for reporting, based on a session timeout value and a padding time. For information about how to set these values, see Viewing product information. For additional details see LevelBlue Knowledge Base article Q11755.
  • Each time a page is requested, WebMarshal checks for a previous request from the same user.
  • If the previous request from the user was within the session timeout value, the WebMarshal logs show a continuous period of browsing.
  • If the previous request was longer ago than the session timeout value, the WebMarshal logs show that browsing activity stopped after the previous request plus padding time, and started again with the new request.

​
Identifying web content using TextCensor scripts

TextCensor scripts allow you to check for the presence of particular content in a file or text. You can apply TextCensor to any text, including web pages, plain text files, and text extracted from PDF or Office documents. TextCensor supports Unicode and wide characters, and is designed to work intuitively with a wide variety of languages. A script can include many conditions based on text combined using logical and positional operators. Triggering of the script is based on the weighted result of all conditions. TextCensor scripts are used in Content Analysis rules. The rule can block a request, classify a site, or add a site to a URL Category. To see a list of rules that use a script, view the properties of the script.

​
TextCensor elements

TextCensor scripts contain one or more expressions, each consisting of a word or phrase. You can use two wildcard characters, anywhere in a word or phrase.
  • * matches zero or more letter or digit characters.
  • ? matches one letter or digit.
Wildcards match only letters and digits, and apostrophes or hyphens that are treated as part of words (see Word breaks). Wildcards do not match other symbol characters. If you want to set the order of evaluation of a complex expression that uses more than one operator, use parentheses ( ). Each TextCensor expression can include logical and positional operators. The operators must be entered in UPPERCASE.

​
Positional operators

TextCensor works with the positions of words or phrases within a file. For example, in the sentence “The quick brown fox jumps over the lazy dog” the word “quick” starts and ends at position 2, and the phrase “jumps over” starts at position 5 and ends at position 6. A positional operator works with expressions that evaluate to sets of positions. It takes two sets of positions as parameters, and returns a new set of positions.
In a simple TextCensor expression, you can think of the expression result as “true” or “matched” if the word or phrase is found in any position in the text. When the word or phrase is found in more than one position, this counts as more than one match of the expression.When you combine positional operators to make a complex expression, note the explanations of the sets returned by each operator (see below). Test your script before applying it in production.
You can specify a distance for many positional operators. The default distance (if you do not specify a value) is 4. Table 5: TextCensor positional operators

​
Logical (Boolean) and special operators

A logical operator takes Boolean (true/false) values as input, and returns a Boolean result. These results cannot be used as parameters of a positional operator. When one of the parameters to a logical operator is an expression that returns a position set, the parameter is treated as a logical value. A set with at least one position match is treated as true. A set that has no matches is treated as false. TextCensor also supports the special operator INSTANCES. Table 6: TextCensor logical and special operators

​
Anchored regular expressions

TextCensor supports use of Regular Expressions through the ARX operator. An anchored regular expression is a regular expression (regex) which must be preceded by a word on the left hand side of the ARX operator. Matching of the regex begins at the next character following the word. Regex patterns must always begin by matching one or more non-word characters. In most cases you can start the regex pattern with \W to match non-word characters.
  • ARX is based on Google RE2. The syntax generally follows well known Regular Expression syntax.
  • Distance parameters for ARX operators are specified in characters.
  • Regular expressions are case insensitive by default. You can force case sensitive matching with the operator ?-i
  • ARX does not support lookahead or lookbehind.
  • ARX does not support capture groups. Capture groups will be ignored or converted to non-capturing sequences.
  • ARX does not support \Q...\E literal text.
  • For further details of the ARX syntax, see the RE2 wiki.
Table 7: TextCensor regular expression operators Multiple anchored regular expressions can be combined with other expressions and operators to create complex statements. For example,
matches the phrase: dog chasing one or more big white cats.

​
TextCensor concepts

The following concepts clarify how TextCensor expressions are evaluated.
This section does not apply to Regular Expression patterns.

​
Words

A word is made up of one or more letters and digits, and sometimes symbols.
  • In alphabetic languages, a word is a group of letters or digits separated by other characters (such as punctuation, other symbols, and white space).
  • In Chinese, or Japanese kanji, a word or “token” may be composed of one or more characters (ideographs).

​
Phrases

A phrase is made up of a series of words separated by word break characters.

​
Symbols and punctuation

Symbols other than letters and digits are not treated as part of a word unless they appear in the specific statement being evaluated. A group of symbols is not treated as a word.
  • The text word$deed is matched as two words by the expression word FOLLOWEDBY deed, and also by the exact expression word$deed
  • The text $word$ is matched by any of word, $word, word$, or $word$
  • The text Save $$$ Now is matched by save FOLLOWEDBY=1 now

​
Word breaks

The sets of characters that are treated as word and number break characters generally follow Unicode standards. A word break character can also be matched exactly or by a wildcard.
  • Each of the following strings is treated as one word:
    • John's
    • 3.14159
    • 1,234.56
    • 3a
    • REV.B (the full stop between letters with no surrounding spaces is not a word break)
  • The text half-baked is treated as two words and is matched by any of the following expressions:
    • half FOLLOWEDBY=1 baked
    • half-baked
    • half?baked

​
Accented letters

TextCensor treats each accented character as a single letter. A letter with additional composed accent characters is normalized to a single character before the text is evaluated.

​
Escape characters

Some characters have special meanings in TextCensor. These characters are parentheses, square braces, the asterisk, the equal sign, the double quote character, and the question mark. You can place a backslash character (\) before any of these characters in order to use the character’s normal meaning. To use a normal backslash character, place two of them together (\\). Within ARX expressions, the Regular Expression reserved characters apply (in particular the forward slash /) which marks the start and end of the regex pattern).

​
Case sensitivity

TextCensor evaluation is NOT case sensitive by default. To perform a case sensitive match, quote the content using double quote characters. All special characters and escape characters retain their meaning within double quotes.

​
Classes

You can use TextCensor Classes to match specific types of characters inside a word, or special types of words. Table 8: TextCensor classes

​
Named statements

You can give a TextCensor statement a name. When a named statement is executed, the result is stored. You can reference it in later statements within the same script. If a statement contains only words or only uses positional operators, the stored result is the set of word positions found by that statement. If the statement uses any other operators then the result is logical. You can reference the result of a statement by using [@name] inside a statement. This can be used anywhere that you would otherwise use the bracketed result of an operator.
Naming a statement does not affect the statement’s score. To use a named statement as a macro expression, in most cases you should set the statement’s score to zero.When using named statements within other expressions, remember that the result must match the required parameter type. If a statement returns a logical result you cannot use it as a parameter to a positional operator. Test your scripts before applying them in production.

​
Scoring a TextCensor script

Each script is given a trigger threshold, expressed as a number. Each expression in a script is given a positive or negative score. If the total score of the content being checked reaches or exceeds the trigger threshold, the script is triggered. The total score is determined by summing the scores resulting from evaluation of the individual expressions in the script. For each expression, if the result is a true logical value, the expression score is the base score. If the expression result is a position set (the word or phrase was found one or more times in the text), by default the final score of the expression is the base score. You can choose how to add the score when the expression is matched more than once. The options are: Table 9: Cumulative scoring options Negative scores and trigger levels allow you to compensate for the number of times a word could be used in text that you do not want to match. For instance: if breast is given a positive score in an “offensive words” script, cancer could be assigned a negative score (since the presence of this word suggests the use of breast is medical/descriptive).
Script evaluation always checks all expressions to obtain the final score. The order of expressions in a script is not significant. This is a change from earlier versions.

​
Adding a TextCensor script

  1. Select Policy Elements > TextCensor Scripts.
  2. Click the New TextCensor Script icon in the tool bar to open the New TextCensor Script wizard. If necessary click Next to continue to the TextCensor Expressions window.
  3. Click New to open the Add TextCensor expression window.
  4. Enter the expression, optionally using the operators described earlier. For example:
    In this example the expression score is added to the script total if the document contains the words dog house (or dog houses, and so forth) in order, and does not contain the word cat.
    TextCensor expressions are not case sensitive by default. However, quoted content is case sensitive. So textcensor would match TextCensor, but "textcensor" would not.
    Add TextCensor Expression window with the expression dog NOT NEAR=2 cat, an expression score of 10, and Add the score for the first match only selected

    Add TextCensor Expression window

  5. Select a score and contribution method for this expression (see Scoring a TextCensor script for more information).
  6. Click Add (or press Enter) to add the expression to this script. The window remains open so you can create additional expressions.
  7. When all expressions have been entered, click Close to return to the New TextCensor Script window.
  8. Select a trigger threshold. If the total score of the script reaches or exceeds this level, the script is triggered. The total score is determined by evaluation of all expressions in the script.
    New TextCensor Script Wizard listing one expression with score 10 and match limit 1, and a trigger threshold of 10

    New TextCensor Script Wizard, Text Censor Expressions window

  9. Click Next
  10. On the TextCensor Script Information page, enter a name and optional description for the script.
  11. Click Next, then Finish, to add the script.

​
Editing a TextCensor script

  1. Select TextCensor Scripts in the left pane.
  2. Double-click the script name in the right pane to open the script properties window.
  3. Double-click an expression to edit it.
  4. To delete a line, select it and click Delete.
  5. Change the script name and trigger level as necessary.
  6. Click OK to accept changes or Cancel to revert to the stored script.

​
Importing a TextCensor script

You can import TextCensor scripts from files.
You can import scripts in the format used by WebMarshal 6.9.5 and above, as well as scripts in the format used by earlier versions. The earlier version scripts will be upgraded to the new format automatically. Any problems with upgrading will be reported.
To import a Script:
  1. Select TextCensor Scripts in the left pane.
  2. Click the New TextCensor Script icon in the tool bar to open the New TextCensor Script wizard.
  3. On the TextCensor Expressions window, click Import.
  4. Select the file you wish to import, and click Open.
  5. Complete the Wizard to add the script.

​
Exporting a TextCensor script

You can export TextCensor scripts to XML or text files. To export a Script:
  1. Select TextCensor Scripts in the left pane.
  2. Double-click the name of the script you want to export in the right pane to open the script properties window.
  3. Click Export.
  4. Enter the name of the file you want to create, and click Save.
  5. In the script properties window, click OK.

​
Using TextCensor effectively

The effective use of TextCensor scripts depends on understanding how the Text Censor facility works and what it does. TextCensor evaluates rules against plain text or HTML documents. The rules can be used to block a request, classify a site or add it to a URL Category. If a Content Analysis rule includes a “block” action, TextCensor scripts are evaluated before the material is returned to the user. Blocking does not apply to content cached on the local computer.

​
Constructing TextCensor scripts

The key to creating good TextCensor scripts is to enter words and phrases that are not ambiguous. They must match the content you want to block. Also, if certain words and phrases are more relevant to the match than others, those words and phrases should be given a higher score to reflect the greater relevance. In creating TextCensor scripts, you should strike a balance between overly-general and overly-specific. For instance, suppose a script is required to check for sports-related sites. To enter the words score and college alone would be ineffective because those words are likely to be used on non-sports sites. Hence, the script would trigger too often, potentially stopping access to acceptable sites such as general news sites. The same script (to find sports-related sites) would be better constructed using the phrases extreme sports, college sports and sports scores as these phrases are sport specific. However, using only a few very specific terms may mean that the script does not trigger often enough. Again using the sports example used above, the initials NBA and NFL, which are very sports specific, should be given a suitably higher score (that is, promoting earlier triggering) than, for example, college sports.

​
Decreasing unwanted triggering

TextCensor scripts might trigger on pages that are not obviously related to the content types they are intended to match. To troubleshoot this problem:
  1. Use the problem script in a TextCensor Rule that classifies sites and adds them to a URL Category, such as “suspected sports sites.”
  2. After using this rule for a while, check the sites that have triggered the script and determine which ones are triggering it falsely.
    • In the URL Categories display of the console, double-click a URL to view the reasons it was added (for example, TextCensor details).
  3. Revise the script by changing the score or key words, so as to decrease false triggering.
  4. When satisfied, create a Standard Rule which denies access to sites in the URL Category generated by the script, and/or add a Block action to the original TextCensor rule.

​
Testing TextCensor scripts

To test the operation of a TextCensor script:
  1. Click Test on the New TextCensor Script or script properties window to open the Test TextCensor window.
  2. In the Test TextCensor window, enter the sample text you want to test using one of two methods.
    • Select Test script against file. Enter the name of a file containing the test text (or browse using the button provided).
    • Select Test script against text. Type or paste the text in the field.
  3. Click Test. The result of the test (including details of the expressions which triggered and their scores) displays in the Test Results pane.
    Test TextCensor Script window testing sample text, with the result Script was triggered scoring 10

    Test TextCensor Script window

You can also test a script as part of the test of a content rule. This method allows you to test using pages drawn directly from the Web. See Testing access policy for detailed information on Rule testing.

​
Using malware scanning

WebMarshal can invoke third-party scanners to check file uploads and downloads for malware, including viruses and other malware. Before you enable scanning Rules, you must install at least one supported scanner on each processing server, and configure the scanners within WebMarshal.

​
Scanning overview

WebMarshal currently supports only specific malware scanners that have licensed DLL interfaces. Supported scanners include Bitdefender for Marshal, McAfee for Marshal, and Sophos for Marshal.
The Sophos (SAVI interface) and Symantec scanners that were previously supported are not supported in this release for technical reasons. Kaspersky for Marshal is no longer sold.
You choose which files to scan using Malware Scanning Rules. See Chapter 6, Understanding web access policy, rule containers, and rules. For enhanced protection against viruses and malware, TextCensor and file type rules should also be used to control potentially dangerous file types such as VB Script and executable files.
WebMarshal uses a temporary directory during scanning. This directory must be excluded from on-access or resident virus scanning. If it is not excluded, the WebMarshal Engine and/or the WebMarshal Controller service may be unable to start. By default, WebMarshal uses the \temp subdirectory of your install directory. You can change this location by editing XML configuration files on each processing server and restarting the WebMarshal services. If you change the location of the temporary directory for either or both services, be sure that you also update virus scanner exclusions.
To view and add to the list of configured scanners, select Malware Protection from the left pane of the WebMarshal Console.

​
Adding a scanner

To add a malware scanner to the list of configured scanners:
  1. Select Malware Protection from the left pane of the WebMarshal Console.
  2. Click the New Malware Scanner icon in the tool bar to start the New Virus Scanner Wizard.
  3. The Select Scanner page of this Wizard shows a list of scanners WebMarshal can use. To obtain more information about any scanner, select it and then click Visit Web Site.
    New Virus Scanner Wizard listing Bitdefender for Marshal, McAfee for Marshal, and Sophos for Marshal

    New Virus Scanner Wizard, Select Scanner window

  4. Select a scanner to add. All scanners that you add will be available for use by WebMarshal. When you create a malware scanning rule, you can choose the scanners that rule will use. You can use multiple scanners in a single rule or separate rules. Because different products have differing coverage, some sites choose to use more than one scanner.
  5. Click Next to continue to the next page.
  6. If any additional parameters are required, the additional parameters pages of the wizard is shown. Enter any required parameters (such as the location of a scanner if it is installed remotely). Click Next to continue to the next page.
  7. Click Finish to install the scanner and exit the Wizard.
To select an additional scanner for use, re-run the Wizard.
Bitdefender for Marshal, McAfee for Marshal, and Sophos for Marshal each require installation of a configuration Console, available in separate downloads from LevelBlue (and licensed separately).You must install this software on the WebMarshal server. If you have configured an array of servers, you must install the scanning software on each processing server.WebMarshal trial keys enable all of these products for the 30 day trial period. To obtain a permanent key, contact your LevelBlue supplier. If you are a customer with a permanent WebMarshal key and you want to try one of these scanner products, contact your LevelBlue supplier for a special time-limited key.

​
Deleting a scanner

To delete a configured scanner from the list of scanners WebMarshal can use:
  1. Select it in the right pane of the Console
  2. Click the Delete icon in the taskpad tool bar.
If any malware scanning rule is enabled, you cannot delete all scanners of the type(s) used by that rule.
  • If any malware scanning rule (including disabled rules) references a specific scanner, you cannot delete that scanner.
  • Deleting a scanner from the list does not uninstall the scanning software.

​
Testing scanners

To test the operation of an installed malware scanner:
  1. Select it in the right pane of the Console
  2. Click the Properties icon in the taskpad toolbar.
  3. Select the Installation Status tab. WebMarshal queries each processing server and returns the status of the scanner on each server.

​
Logging activity with classifications

WebMarshal logging classifications allow you to record more detailed information about user requests (both allowed and denied), and content downloaded or uploaded. Logging classifications are only recorded in database logging.

​
Types of logging classification

WebMarshal allows you to use Classifications in two ways.
  • Domain Classification actions are available within Standard rules and TextCensor rules. Logging a classification for a domain shows that a user browsed to a URL which is in a specific category, or to a page which triggered a rule. A domain can receive multiple classifications within a single browsing session. A domain could also receive different classifications in different sessions, depending on the actual content requested, such as sports or entertainment sections of a news site.
  • File Classification actions are available within all Content Analysis rules as well as download file type/size rules and malware scanning rules. A file classification applies to a specific upload or download request.
You can generate reports by user or domain based on the classifications recorded. For details see Chapter 8, Reporting on browsing activity. In addition, when any filtering list integration is enabled, every browsing request returns a categorization as provided by the filtering list. These categorizations are recorded regardless of whether they are used within rules to filter requests. This process is separate from the logging of classifications. You can generate reports based on the categorizations. To see a list of rules that use a classification, view the properties of the classification.

​
Adding a logging classification

To add a classification:
  1. Select Classifications in the left pane of the console.
  2. Click the New Classification icon in the tool bar to open the New Classification wizard. In the New Classification window, enter a name and optionally a description for the classification.
    New Classification Wizard with fields for the classification name and an optional description

    New Classification Wizard, Classification Information window

  3. Complete the wizard to add the classification.

​
Editing a logging classification

To edit a classification:
  1. Double-click the classification name in the Console to open the Edit Classification window.
  2. Make any required changes.
  3. Click OK.

​
Deleting a logging classification

To delete a Classification:
  1. Right-click the classification name in the Console.
  2. Select Delete.

​
Notifying users with notification pages

WebMarshal uses web pages to provide pre-configured notification messages to users. The notifications can also be customized.

​
Notification web pages

When a user requests a Web resource that triggers a rule, they may receive a notification web page. The notification could simply state that a request was denied, or it could request acknowledgement of a warning before processing the request. The choice of notification pages is made on the Action page of each rule wizard. Choices vary according to the type of rule.

​
Default notification pages

WebMarshal is supplied with a number of default notification pages. The files are standard HTML pages, located in the \ArrayManager\Policy\Templates sub-directory of the WebMarshal installation directory on the Array Manager server.
When you commit WebMarshal Configuration, any changes to items in this folder are copied to each processing node server, and stored in the folder Node\Policy\Templates. New files and subfolders are also copied.You should only make changes to the Array Manager folder. The Node folders will be overwritten.You can store small helper files in this location, but any large content should be served from another location.
  • Page Blocked: for denied web pages and other items where no part of the file was returned to the browser.
    • Standard
    • AdvertisingSmall (used in default rules for advertising sites)
    • Blank (used for images)
    • Offensive (for offensive content)
    • Small
    • With Sound (audible warning)
  • File Aborted: for denied file type or size, where a portion of the file was returned to the browser.
    • Standard
    • Malicious (Malware content)
    • Virus (virus content)
  • File Blocked: for denied file type or size.
    • Standard
    • Malicious (Malware content)
    • TextCensor (lexical scan found denied content)
    • TextCensor Offensive (lexical scan found offensive content)
    • Virus (virus content)
  • GoogleWebRisk: used by the Google Web Risk policy. Any custom versions of these pages must include the Google disclaimer.
    • GoogleWebRiskBlocked
    • GoogleWebRiskWarning
  • Quota: for quota rules.
    • Quotas (displays the user’s quota)
    • Quota Exceeded
    • Quota Extend (allows the user to request an extension)
  • TRACEnet: used by the TRACEnet policy.
    • TRACEnet Blocked
    • ReclassifyThanks (acknowledges the user’s request to reclassify a blocked item)
  • Upload Blocked: for upload rules.
    • Standard
    • Malicious (Malware content)
    • TextCensor (lexical scan found denied content)
    • Virus (virus content)
  • Warning: before access to “permitted with warning” sites.
    • Standard
    • Malicious (Malware content)
    • Offensive (lexical scan found offensive content)
    • Policy (for policy display rules)
    • Previous Scan (site was placed in a URL category by a previous scan)
    • Quotas (states that access is subject to quota)
    • TextCensor (lexical scan found questionable content)

​
Editing notification pages

You can create new versions of the warning/blocking pages, or modify the existing pages. Use HTML editing software to create or modify warning/blocking pages. There are several types of warning/blocking pages, each identified by a filename prefix. New page names must start with one of the defined prefixes. The page types are as follows:
  • Aborted: used when blocking websites where a portion of the file has been returned to the browser.
  • Blocked: used when blocking websites.
  • Warning: used to prompt the user before permitting access to a site.
  • FileBlocked: used when a file is blocked by a file type or malware rule.
  • UploadBlocked: used when a file upload is blocked.
  • QuotaExceeded: used when a WebMarshal quota has been exceeded by the user.
  • QuotaExtend: used to allow the user to extend a quota.
For example, to create a new blocking page for pornographic content, copy the file Blocked.htm to a new file BlockedPorn.htm. From then on, BlockedPorn.htm is available in the list of blocking pages when editing a site rule. Please note the following important recommendations before editing the warning/blocking pages.
  • Do not change the original page; instead, copy the file and make any changes to the copy.
  • After changing pages, test them to make sure that they are still functioning correctly. Several pages include special forms that process user input; amending such a page may affect the prompting functionality.
  • If you want to refer to the \Templates sub-directory, use the virtual domain name “webmarshal.home.” For example, a link to a file placed in the \Templates sub-directory could be:
  • Files over 32KB in size (such as a company policy document) should not be served from the Templates sub-directory, to avoid deadlock problems with configuration updates. Host large files on another server/site.
  • If you make links to other sites, ensure that these links are full URLs, not relative paths. WebMarshal replaces the content returned from websites with the warning/blocking pages. If you use a relative path, the link will refer back to the original blocked site.
You can also customize error pages for proxy authentication and other system functions. Please see LevelBlue Knowledge Base article Q10318 for more details.