Troubleshooting one on one

During the past few weeks I’ve collected a set of tools which assist in troubleshooting XenApp and XenDesktop orientated architectures. Before you continue make sure you have a look at the Citrix Brief Troubleshooting Guide: support.citrix.com/article/CTX106727 and the Citrix Logon Optimization Guide: support.citrix.com/article/CTX128277. Both are a MUST read with lots of tips and tricks, what and when to ask, and dozens of knowledge base articles including explanations on the logon processes etc…

Although the above documents also target some of the older XenApp and XenDesktop releases they are still great resources to start out with, remember that the basics (almost) never change and understanding them is an important step in solving whatever issue you may run into.

Citrix Diagnostics Toolkit

This one was brought to my attention recently (added on 02-09-2013) and it’s a must have! See for yourself: support.citrix.com/article/CTX135075 although it pretty much has it all, please continue reading ;-)

Citrix Quick Launch Tool

It’s primary focus is to test and initiate ICA sessions letting you choose between different connection types as well as XML application enumeration and XenDesktop connectivity. There are tons of configuration possibilities as you can see here:

Check: support.citrix.com/article/CTX122536. Also a great tool for troubleshooting Speed screen and display related issues. It is important to note that the Quick Launch Tool isn’t supported by Citrix Technical support, although that doesn’t stop you from using it to test connections and so on.

Seamless Settings Configuration Helper Tool

For those of you familiar with the so called applications flags (to be honest, I’m not that familiar with the material myself). The tool is divided into three main areas: Global flags, Per application flags and Additional settings not application specific. It allows you to back-up and restore settings applied so any step taken can be rolled back, ideal for troubleshooting purposes. Citrix support page: support.citrix.com/article/CTX115146.

Citrix group policy Modeling Wizard

Citrix policies have evolved over the years. Nowadays XenApp 6.5 farms are almost completely managed through policies with only a few exceptions. At first we only had the IMA based policies, as of from XenApp 6.0 we can also configure Citrix policies using our existing Active directory infrastructure, just make sure you install the XenApp bits on the server from where you will run the GPMC. This is done by installing the Citrix Delivery Service Console or App Center software. This will add an extra Citrix ‘container’ in your GPO’s so you can easily configure Citrix specific policies from within AD a.k.a. Citrix AD policies. Last but not least we have the ‘normal’ Microsoft AD policies which have been around for years.

As far as Farm policies go Citrix recommends using one of two methods, so just IMA based or Citrix AD policies only, not both.

The modeling wizard is known by most but often over looked, it tells you which policies would, or perhaps better said, should be applied. You can apply filters like Worker Groups and Access Control tags to narrow your search. When using both AD (Citrix and normal) and IMA based policies run this tool from within the App Center, this way it will display both AD and IMA based policies combined which will give you an clear overview.

GP result

This tool gives you an overview of what is actually being applied. Note that when running GPResult or RSOP from the command-line it will not show any Citrix AD policies being applied on a Citrix XenApp 6 server, this is an know issue. Instead you must run the GPResult from within the GPMC by right clicking Group Policy Results and select the Group Policy Results Wizard. Make sure that the XenApp bits are installed on the server from where you run the GPMC, see my comments on this earlier.

Some more detail

AD policies, either normal or Citrix, by design take precedence over IMA based policies and could potentially overwrite certain settings. Although this sounds straight forward it can be a bit tricky, I’ll try and explain why, with the emphasis on try ;-)

Let’s say you configure an IMA based policy with clipboard mapping enabled, you also configure an Citrix AD policy in which you disable clipboard mapping. Here’s what will happen: The IMA based policy will be applied first enabling clipboard mapping, next the Citrix AD policy gets applied disabling clipboard mapping and will leave it this way. This is normal behavior because both are Citrix policies, one overwriting the other.

But wait… instead of an Citrix AD policy we now configure an normal AD policy and disable clipboard mapping. You would expect the same result as above because of the AD policy precedence, but no. In this case the normal AD policy does get precedence over the IMA based policy but it won’t change the Citrix specific setting, clipboard mapping will stay enabled as configured in the IMA based policy. When a normal AD policy has the exact same settings configured as an IMA based or Citrix AD policy it cannot overwrite a Citrix policy. Fortunately this will not happen that often.

Although it cannot directly overwrite an Citrix AD or IMA based policy it could interfere if different settings are configured that both target the same outcome, again an example:

You configure an Citrix AD or  IMA based policy in which you specify an Server idle timer interval of some sort, this is not a setting you can configure in a normal AD policy, for this you would need the XenApp bits installed as explained earlier. But you could configure an normal AD policy in which you specify a Remote Desktop Services idle session limit. Both policies, although configured differently, achieve the same result. If in this case you configure the Citrix specific policy to 5 minutes and the normal AD policy to 10 minutes, then 10 minutes it is! Here’s the example I used: support.citrix.com/article/CTX134756.

Does this make any sense?! :-)

Session Sharing, CDF control: Citrix Diagnostic Facility

CDF allows various components and processes of XenApp to be traced, such as the Citrix Print Manager Service, Smart Cards, and Client Drive Mapping, to name a few. Make sure you have the following information at hand when analyzing the trace: User name, client name, server and session ID, this to make browsing the logs easier. You can download it here: support.citrix.com/article/CTX111961.

Each time a session is started on a XenApp server the following unique registry entry will be made for that particular session: HKLMSOFTWARECitrixIcaSession[n]Connection where [n] is the session ID. This key potentially holds valuable information regarding session sharing, when multiple application parameters are present it indicates that session sharing is successful. Each application can have its own application state indicated by a number:

0 = Published desktop or ICA listener both cannot be shared, 1 = Session is Active, 2 = Pre launch session, 3 = Lingereing session and 4 = Application not running.

XDPing

One of the most common XenDesktop issues is a VDI showing as unregistered on the DDC, XDPing can help detect registration and other problems in different ways, it inventories:

1. Information and status of NIC interfaces and network settings.
2. Performs DNS lookups and reverse lookups on the IP address of the device.
3. Time synchronization information and time check for Kerberos authentication.
4. User information for logged in user.
5. Machine information.
6. Information on XenDesktop services.
7. Information on firewalls.
8. Queries the local event log to check for known events that are related to XenDesktop.
9. Provides client bandwidth and response time information from the VDA to the client.

Citrix Scout

Can be used for both XenApp and XenDesktop architectures, as per Citrix: Run Citrix Scout from a single XenDesktop controller (DDC) or XenApp server (Data collector) to capture key data points and CDF traces (you can run multiple at once) for selected computers followed by a secure and reliable upload of the data package to Citrix Technical Support. It collects information on events, registry, DLL versions, CFD traces on multiple devices. Detailed information: support.citrix.com/article/CTX130147.

Citrix MedEvac for XenApp 5.0 and down

As per Citrix: This tool can be used to check the following components of a XenApp farm:

XML Service health:

1. Verifies that the XML Service is able to respond to an XML, Web Interface, XenApp Plugin or Program Neighborhood requests.
2. Verifies that the XML Brokers are able to contact the data collector.

Verify data collector health:

1. Verifies data collector is able to provide a least-loaded server.
2. Verifies the IMA Service on the data collector is functioning properly.
3. Verifies the IMA Service can read the data collector’s local host cache.
4. Verifies that the IMA Service can read its dynamic store data.
5. Verifies at least one server in the farm has this application published.

Least-loaded server health:

1. Verifies Terminal Services on the least-loaded server.
2. Verifies the Remote Procedure Call (RPC) Service on the least-loaded server.

Netdiag

As per Microsoft: A command-line diagnostic tool helps to isolate networking and connectivity problems by performing a series of tests to determine the state of your network client.

XD Collector

Does basically the same as Citrix Scout with the difference being that it has a different GUI and can only be used for XenDesktop.

NSLOOKUP

You can use the Nslookup.exe command-line tool to perform query testing of the DNS domain namespace and to display configuration information.

FC.exe

Can be used to compare two files. Use it to compare Ntuser.dat files when troubleshooting logon issues, corrupt profiles etc…

Citrix specific Command-Line parameters

ICAPORT

Query or change the TCP/IP port number used by ICA. Default port is 1494, the port range must be somewhere between 0 and 65535 and must not be in conflict with other well know port numbers.

TWConfig

Use twconfig to configure ICA display settings that affect graphics performance for clients.

AUDITLOG

This tool is available with the XenApp installation. It generates reports of logon and logoff activity based on the server security event log. You must first enable logon and logoff accounting.

PRINTUI

Built-in utility to perform many printing related tasks including pushing print drivers, adding and deleting printers, and more.

CLTPRINT

Use cltprint to set the number of printer pipes for the client print spooler.

Citrix specific Query commands

QUERY SERVER – View Citrix Servers.

QUERYDS – Query data on the local zone data collector.

QUERYDC – Determine the data collector for a given zone.

QUERY SESSION – Displays information about sessions running.

QUERY ACL – Security Audit Utility.

QUSER – Queries connected users.

QUERY LICENSE – View Citrix Licenses.

QUERY SERVER – View Citrix Servers.

QUERYHR – Displays information about member servers in the farm.

QUERY PROCESS – Displays information about processes running on a terminal server.

QUERY TERMSERVER – Displays a list of all terminal servers on the network.

QUERY FARM – Displays information about servers within an IMA based server.

QFARM – Same as above.

Divers Citrix specific commands

ACLCHECK – Security Audit Utility.

CHFARM – Use CHFARM to move a server from its current server farm.

DSCHECK – Provides a way to perform validation checks on a server farm’s data store.

DSMAINT – Is used to configure the IMA data store database for a server farm.

DSVIEW – Can be used when you need to look at the value in the data store or the LHC.

More?

Hopefully you found something useful?! Suggestions are more than welcome, what kind of tools do you use, if any? What would be your recommendation when the going gets tough? Please do share, I’m sure there must be plenty I’ve missed.

Bas van Kaam ©

Reference materials used: Citrix.com, Brain Madden.com and Google.com

[twitter-follow screen_name='BasvanKaam']

Bas van Kaam

Field CTO EMEA by day, author by night @ Nerdio

Father of three, EMEA Field CTO @ Nerdio, Author of the book Van de Basis tot aan Meester in de Cloud, Co-author of the book Project Byte-Sized and Yuthor of the book: Inside Citrix – The FlexCast Management Architecture, over 500 blog posts and multiple (ultimate) cheat sheets/e-books. Public speaker, sport enthusiast­­­­­­­­: above-average runner, 3 x burpee-mile finisher and a former semiprofessional snooker player. IT community participant and initiator of the AVD User group Community world wide.