In this article I shall look at what to check when your BlackBerry users are unable to either receive mail, send mail, or both.
This article will only examine how to troubleshoot BES when deployed against an Exchange server rather than Domino or Groupwise.

Regardless of which version of the BES software you are running, the way in which it access Exchange is essentially the same - a system account is granted read and write access to user's mailboxes and manages their mailbox on their behalf. In the event of a problem, the specific troubleshooting steps will vary depending on the version of Exchange you are running, but the basic elements to verify are common to all:

• Number of users affected
• Device network connectivity
• User mailbox status
• Message filters
• BES server services
• BES server network connectivity
• BesAdmin Exchange permissions
• Event Viewer
• Log files

This article also assumes that users have already successfully activated their devices and have been able to both send and receive mail before, and have only now started experiencing problems. For details on how to troubleshoot the Enterprise Activation procedure, view this article - http://blog.brightpointuk.co.uk/how-does-bes-wireless-activation-process...

This article is not intended as an exhaustive troubleshooting guide. Should all of the steps outlined in this article appear to be passed by your BES deployment, and yet you continue to experience difficulty, then detailed examination of the BES log files and use of the BES Resource Kit may be required to isolate and rectify the cause of the problem. Brightpoint's technical support staff are fully-versed in all aspects of the BES solution and able to provide expert support.

BlackBerry Architecture

When troubleshooting BES issues, it is important to understand how the end-to-end BlackBerry architecture works and the components involved.

RIM deploy and maintain what is referred to as the BlackBerry Infrastructure (BBI). BES servers are assigned unique SRP IDs, which are used to authenticate against and register with the BBI using the Server Router Protocol (SRP). Connections are initiated from the BES to the BBI on a single TCP port, number 3101. This is the reason why BES servers are not required to be "Internet-facing" and do not require public IP addresses.
Client devices also register with the BBI, using unique PIN numbers. Messages sent from the BES are encrypted and then routed over the Internet to the BBI, including the target device's PIN number in the header information. Similarly, messages sent from handhelds are encrypted and sent to the BBI using the BES server's unique SRP key in the header.

Typically, the ability to receive email but not send is either due to Active Directory permission issues, or an exception to the Exchange mailbox size limit policy. If mail is being received then it is relatively safe to assume that both client device and BES are operating correctly. If mail is not being received (or sent), then a sensible troubleshooting procedure would include the following.

NOTE - the same troubleshooting procedure applies to the BlackBerry Professional Software (BPS).

Number of users affected

If only one user is reporting problems, then it is advisable to verify the status of the client device and that specific user's Exchange mailbox before you worry about the server. If all users, or an administratively-significant number of users are reporting problems then the server status should be examined. As the administrator, if you have a device yourself, is yours working?

Device Network Connectivity

If you believe the issue to be unique to a specific user, then verify that the device itself is correctly configured.

Is the cellular element of the device enabled? By default when powering on devices all networks are disabled. If the device has been used in a location that requires cellular devices be deactivated, has the user remembered to reactive the radio?

Can the device be used to make voice calls (if voice is enabled on the SIM and the IT policy allows use of the phone)? If the user receives a recorded message indicating that a call to customer services is required....then a call to customer services may be required to rectify an account issue.
Is the device registered with the cellular operator and indicating a packet data connection (either "GPRS", "EDGE" or "3G")? If not, the user may not be in a suitable coverage area or the operator may be experiencing a localised network fault.
Does the user definitely have the BES subscription on the SIM they are using in the BlackBerry device?

To force a device to register with the BlackBerry Infrastructure, select Menu --> Options --> Advanced Options --> Host Routing Table --> Menu --> Register Now

In many cases, simply removing the battery from the device and reinserting it after a couple of seconds may resolve the issue.

You can verify that the device is correctly registered on both the cellular network and with the BlackBerry infrastructure by sending a PIN message from the device to itself: within the Inbox on the device select the option to Compose PIN and send a message to yourself. This will cause a message to be sent from the device back to itself via the BlackBerry Infrastructure independently from the Exchange server.

New to version 5.0 of the device handheld software is a mobile network diagnostic test utility. Should you want to perform more detailed diagnostics, select the Options icon from the main menu. Select the Mobile Network menu entry:

The following screen will be displayed:

Press the menu button and select the entry for Diagnostics Test. The following window will be displayed:

Press the menu button and select the Run option:

Verify that all tests are completed successfully:

• BlackBerry Registration
• Connected to BlackBerry
• BlackBerry PIN-PIN

User Mailbox Status

If the user's device appears to be operating correctly, can the user access their mailbox via Outlook or Outlook Web Access? If the user's account has been disabled for any reason the administrator will need to ascertain why and whether the account can be reactivated.
If the user has exceeded their Exchange mailbox size limit, the ability to receive new mail may have been disabled and the user will need to free up some space in their mailbox before any new mails will be delivered.

Message Filters

If a user is receiving some email messages but not all, or is not receiving any messages but troubleshooting so far indicates that everything appears to be functioning correctly, verify what messages filters have been applied to the user's account: it may simply be the case that the user has inadvertently applied a rule that prohibits certain or all mails from being forwarded to their BlackBerry device:

BES services

Should more than one user have reported problems, and your own device not be receiving mail, verify the status the BES. Ensure that all required BlackBerry services are running and check the Application log in the Event Viewer for any warning or critical error messages.
Restarting the BlackBerry Controller service will cause all BlackBerry services to be refreshed.

BES network connectivity

If no messages are being received by handhelds, either email or PIN messages, verify that the BES is connected to the BlackBerry Infrastructure. You can verify whether the BES or BPS server is successfully connected to the SRP infrastructure by launching the BlackBerry Server Configuration utility from the Start menu and selecting the BlackBerry Router tab:

Click the Test Network Connection button and verify that the test is successful. Should the test fail verify that the server has outbound Internet access on TCP port 3101 on any firewalls between the BES and the outside world.

Alternatively you can run the bbsrptest.exe utility at the command line. On the BES server open a command prompt and navigate to the directory where the BES software has been installed to - c:\Program Files\Research In Motion\BlackBerry Enterprise Server\Utility by default.
Run the following command:

bbsrptest.exe

If the test is successful the following will be displayed:

Alternatively, if you have a BlackBerry Technical Support subscription, you can view the status of a specific BES SRP key and re-enable it should it have been disabled for any reason:

Verify whether the company Internet feed may be congested or have insufficient bandwidth to accommodate the volume of traffic being generated using SNMP monitoring tools such as MRTG (http://blog.brightpointuk.co.uk/mrtg)

Should the SRP connectivity test be successful, you can also verify end-to-end connectivity between the BES and the handheld by sending the user a PIN message from the BES:

BesAdmin Exchange permissions

Should PIN messages be delivered end-to-end successfully, then this would indicate that both BES and devices are operating correctly. Should Exchange emails not be being delivered, then the permissions of the BesAdmin account (or the user account used to install the BES server) should be verified. This step is essential when troubleshooting issues whereby emails are being received by users, but no emails are able to be sent.

In any Exchange BES installation, the BesAdmin will need the following rights in order to be able to both send and receive emails successfully:

• Exchange View Only Administer privileges
• "Send As", "Receive As" and "Administer Information Store" privileges on the Exchange server
• "Send As" permissions on User Objects on the Active Directory domain

The steps to verify that these permissions have been correctly applied are as follows:

Exchange 2003

Delegate administrative control to the BesAdmin user account

On the Exchange server, select Start → Programs → Microsoft Exchange → Exchange System Manager.

Right click on the Organisation name (at the root of the directory in the left-hand pane) and select Properties. The following window will be displayed:

Tick the options to Display routing groups and Display administrative groups. Click OK. If you receive a warning message indicating that the Exchange System Manager needs to be closed and re-opened for the changes to take effect, click OK.

Close the Exchange System Manager and then re-launch it again from the Start menu.

Right click on the first Administrative Group and select Delegate Control, as shown below:

The Administration Delegation Wizard will be displayed:

Click Next. The following window will be displayed:

Click Add. The following window will be displayed:

Set the Look in field to the domain in which the BES server resides. From the list of users select the BesAdmin account and click OK.

Click Next and then click Finish to complete the wizard.

Assign Send as, Receive as and Administer information store rights to the BesAdmin user account

Within the Exchange System Manager, right click on the entry for the Exchange server which the BES is going to communicate with and select Properties, as shown below:

Click on the Security tab. In the list of users select the BesAdmin user account. In the list of permissions, scroll down and tick the options to allow Administer information store, Receive As and Send As.

Click OK.

For full details on how to prepare Exchange 2003 for a BES installation, read this article - http://blog.brightpointuk.co.uk/bes-41x-50-exchange-2003-pre-requisites

Exchange 2007

Assign Send as, Receive as and Administer information store rights to the BesAdminuser account

Unlike previous versions of Exchange. This needs to be done at the command line via the Exchange Management Shell.

Launch the command interface and enter the following command:

get-mailboxserver (servername) | add-adpermission –user (service account)-accessrights GenericRead, 
GenericWrite -extendedrights Send-As, Receive-As, ms-Exch-Store-Admin

Where (servername) should be replaced with the name of the Exchange Server, and (service account) should be replaced with the Alias name of the BesAdmin user account (so ‘BesAdmin’ in this case)

If you are successful, you should see the following:

To verify the permissions of an existing account, type:

get-mailboxserver (servername) | getADpermission -user (service account) | Format-List

For full details on how to prepare Exchange 2007 for a BES installation, read this article - http://blog.brightpointuk.co.uk/bes-41x-50-exchange-2007-pre-requisites

Exchange 2010

Assign "Receive As" and "Administer Information Store" rights to the BesAdmin user

On the Exchange server, launch the Exchange PowerShell and issue the following command:

Get-MailboxDatabase | Add-ADPermission -User "BesAdmin" -AccessRights
 ExtendedRight -ExtendedRights Receive-As, ms-Exch-Store-Admin

Assign Exchange View-Only Administrator rights to the BesAdmin user

Still within the Exchange PowerShell, now issue the following command:

Add -RoleGroupMember "View-Only Organization Management" -Member "BesAdmin"

Assign "Send As" rights to the BesAdmin user

This is done on the Exchange server itself within the Exchange PowerShell. Launch the console and issue the following command:

Add-ADPermission -InheritedObjectType User -InheritanceType Descendents -ExtendedRights
Send-As -User "BesAdmin" -Identity "CN=Users,DC=domain,DC=com"

(where "domain" and "com" should be substituted for your specific domain details, eg: DC=brightpoint,DC=co,DC=uk and so on)

To force all of the above changes to take effect on the domain, it may be worth running a group policy update. On the Exchange server click Start --> Run and issue the command "gpupdate /force"

Turn off Exchange 2010 Client Throttling

Exchange 2010 uses client throttling by default to protect the Exchange server from excessive user demands. RIM recommend turning off this feature as it can have an adverse affect on the performance of the BES solution. This is done within the Exchange PowerShell console.
Launch the console and issue the following command to get the "Identity" of the default throttling policy"

Get-ThrottlingPolicy | Where-Object {$_.IsDefault -eq "True"} | FL Identity

the Identity will be displayed:

Now issue the following command:

Set-ThrottlingPolicy -RCAMaxConcurrency $null

You will be prompted to enter the Identity to apply the policy to, enter the result returned above:

Increase the maximum number of connections to the Exchange Address Book Service

On the Exchange Server (or specifically the Client Access Server in a multi-box deployment), browse to C:\Program Files\Microsoft\Exchange Server\V14\Bin and locate the file "microsoft.exchange.addressbook.service.exe.config" and open it in NotePad:

Locate the line "MaxSessionsPerUser":

Increase the value to 100000. Save the file then restart the Address Book Service:

For full details on how to prepare an Exchange 2010 environment for a BES installation, read this article - http://blog.brightpointuk.co.uk/bes-501-exchange-2010-pre-requisites

All versions of Exchange

Assign Send As rights on Domain User Objects to the BesAdmin user account

On the Exchange Server, launch the Active Directory Users and Computers MMC snap-in:

Open the View menu and select the option to show Advanced Features.

Right click on the Domain root and select Properties. Click on the Security tab:

Click on the Advanced button. Select the option to Add a user:

Enter the alias of the BesAdmin account created earlier and click OK. In the Apply Onto drop-down menu select the option for User Objects:

In the Permissions section select the option to enable Send As:

Calendar Issues

Should email messages be sent and received successfully between the BES and handheld devices, but calendar appointments not be synchronised, a common step missed when installing the BES solution is to register the CDO.dll file on the BES server.
This only applies to versions of Exchange prior to 2007. When installing BES against Exchange 2003 and earlier, usual practice is to install the Exchange System Manager on the BES server. This installs the required MAPI components, but does not install the files required for calendaring.
The CDO.dll file needs to be located on the Exchange server, copied to the System32 directory on the BES server and registered:

By default the cdo.dll file will be located in the C:\Program Files\Exchsvr\Bin directory. To check the version of the file, right click on it and select Properties. Click on the Version tab:

The cdo.dll file needs to be registered, otherwise wireless calendar synchronisation will not function correctly between the Blackberry handheld and the server. To register the file, copy it to the C:\WINNT\System32 directory.

Once the file has been copied, select Start → Run. Enter “cmd” in the dialogue and press OK.

A command prompt will be displayed. Change to the WINNT\System32 by typing cd winnt\system32 and pressing enter.

Type regsvr32 cdo.dll and press enter:

If the file is registered successfully, notification will be displayed as shown below:

This procedure is not required for BES installations against versions of Exchange later than 2003 as the MAPI CDO package is used rather than the Exchange System Manager.

Troubleshooting BesAdmin user account permissions

The commonest cause of problems when troubleshooting issues with a BES installation is that the correct permissions have not been assigned to the BesAdmin user on the domain and the Exchange server as detailed above.
Included with the BES 5 software is a utility called "IEMSTEST" which can verify the BesAdmin user's access to specific user mailboxes.

The utility lives in the C:\Program Files\Research In Motion\BlackBerry Enterprise Server\Utility folder and needs to be run at the command line:

Select the BlackBerryServer MAPI profile when prompted:

Select the user account you wish to query:

The permissions will be tested:

As you can see from the above screenshot this test has indicated that the BesAdmin account does not have Send As rights on my James Liddiard user account. Once I verify my permissions, re-running the test indicates that all test have passed successfully:

General Troubleshooting

Should you be unsure as to whether the server running the BES server meets the requirements of the software, or has all the components necessary for integration with Exchange, you can run the BlackBerry System Requirements Tool, part of the BlackBerry Enterprise Server Resource Kit:

This information will be required by Technical Support should you wish to escalate an issue.
The BlackBerry Enterprise Server Resource Kit (BRK) can be downloaded free of charge from the BlackBerry web site.