Ever wondered what happens after a user fills in his or her username and password? What the difference is between internal and external user authentication, resource enumeration and the accompanying launch process? Or what about Web-Interface and StoreFront, do they both have the same authentication mechanism? What are the differences between the server and desktop VDAโs? What about the VDA and XenDesktop internals, what happens inside a VDA when a resource is launched? How does a connection get brokered? Read about this and more during the next 8 (mini) chapters.
Heads up! I recently published version 2.0 of the ultimate Citrix XenDesktop 7.x internals cheat sheet. Click the download button below to get your copy. It contains a ton of additional information on all main FMA components, FMA core services, StoreFront, NetScaler, LHC, desktop and server VDAโs, Receiver, ICA/HDX and more.
When troubleshooting a XenDesktop environment/architecture itโs important to know which components and services are involved, how they interact and what is supposed to happen during normal operations. In fact, I guess itโs safe to state that, that goes for all problems in life, if you donโt know or understand the basics of what you are dealing with than youโre bound to get lost, fast. Throughout this guide Iโd like to zoom in on the FMA, some of its main components and talk about what happens during some of the most common day-to-day operations and processes, so common that in most cases we donโt even think about whatโs going on under the hood untilโฆ itย breaks down!
The way I see it
I am aware that most of the topics, not all though, that Iโm going to discuss are very popular and as such are well documented, either by fellow community members or, in some cases, by Citrix. However, I still feel that Iโll be able to provide you with some new insights on some of the topics covered. Also, and even if itโs only for me, I think itโs nice to have all related information in one place and not scattered over multiple blog posts, which might be considered bad marketing by some, I know :-)
Iโll provide what I prefer to find
Whenever I am on the lookout for new information, whether it is for troubleshooting purposes or just to learn something new, I always hope to find two things… First of all, I hope to find detailed and well written information, and secondly, a way to download the information as a .PDF and/or Word document for future reference. So thatโs what I tried to achieve here as well, hopefully youโll find it detailed (and well written) enough, and next to this blog post I also included the option to download the information in .PDF format. Make sure to check out one of my previous articles as well: Troubleshooting Citrix XenDesktop & XenApp 7.x? Read this! since I feel itโs closely related, enjoy! If you find any typo’s, and I’m absolutely positive that you will, just ignore them!
Letโs take it one step at a time
Iโll try and break it down phase by phase, starting with a brief FMA overview, followed by the user authentication/enumeration process and take it from there. Below you’ll find what I want to cover during some of the next (mini) chapters. It goes without saying that what applies to XenDesktop applies to XenApp as well since they are both part of the same Flexcast Management Architecture.
- Compact overview on the main components that make up the FMA.
- Internal & external (NetScaler Gateway) user authentication steps.
- The application, or resource, enumeration process.
- What happens when a resource, desktop or application, is launched.
- More specific, what happens inside the VDA during launch time.
- ICA protocol stack breakdown, not really me taking but still.
- A look at the main services that make up XenDesktop, or FMA if you will.
- Some key takeaways!
1. FMA high-level overview
Just as a pre-appetizer, letโs have a look at some of the main components that make up the Flexcast Management Architecture (FMA) today and, from a birdโs eye view, how they interact and rely on each other before we get into some more detail on this.
1.1 Receiver
Weโll start with Receiver; itโs installed on the end user device and communicates with NetScaler, StoreFront and the virtual and/or physical machines in the data center over port 80 or 443. This can either be a VDI or HSD orientated architecture. It talks to StoreFront using the StoreFront Service API, again using ports 80 and/or 443. This API is primarily responsible for operations like, user authentication, enumeration, reconnecting, disconnecting, launching applications and desktops, power control and more. There is also a so-called StoreFront Web API a.k.a. Receiver for HTML 5, it provides access to applications and desktops using just a HTML 5 capable web browser, no locally installed Receiver is needed. Both APIโs act as a bridge between the StoreFront services, with the authentication service and the store service being the most important ones.
1.2 StoreFront
Within a XenDesktop Site you basically have two points of authentication, one of which is StoreFront and the other being the NetScaler Gateway. StoreFront communicates with Receiver, the Delivery Controllers and NetScaler (STA) with regards to external access. Optionally it can also be configured to communicate with AppController as part of a XenMobile deployment, VDI-in-a-Box is also an option. It plays an important role in the application enumeration and resource launch (.ICA file) process and it functions as the main Store (there can be multiple) from which users subscribe to their desktops and applications.
1.3 NetScaler gateway
As mentioned above the NetScaler Gateway is primarily used (yes, it can do a lot more, out of scope for now) to provide our users with secure external access to our XenDesktop sites. As we will see shortly there are some key differences when it comes to external and internal user authentication and the resource enumeration process. So in that respect you could say that it is part of the FMA.
1.4 Delivery Controller
Next up is the Delivery Controller; this is the real workhorse and centerpiece of the FMA. It brokers (VDA) sessions, verifies user credentials in AD when combined with StoreFront (note that I say verify and not authenticate, there is a difference between the two) and as such plays an important role during login and resource enumeration. It handles communications with StoreFront and/or Web Interface (XML), the underlying Host (Hypervisor) infrastructure, the central Site database and it also takes care of load balancing HSD connections, and as of version 7.6, it also includes Connection Leasing! You could say that the Delivery Controller is in fact the XenDesktop/XenApp installation. It houses all critical services, which in their turn control all major FMA components. More details in a bit.
1.5 Virtual Desktop Agent
The Virtual Desktop Agent, or VDA in sort, is a piece of software that gets installed on all virtual and physical machines running a Windows Server and/or Desktop operating system in your XenDesktop Site (Citrix recently released a VDA for Linux as well) making their resources remotely available to users. The installed VDA software communicates with the Delivery Controller as well as with the Receiver software installed on the client endpoint.
When the connection is local (LAN) the VDA and Receiver will set up a direct (one to one) connection, if the connection is external, through NetScaler, traffic will flow from the internal network through NetScaler (SSL 443) on to the external client device. Or, if the client endpoint does not have Receiver installed it will, or can, use the StoreFront HTML5 based build-in Receiver instead. The earlier mentioned one to one connection will then run from the users browser (needs to be HTML 5 capable) to the virtual machine in the data center. It will of course traverse the NetScaler Gateway when the connection is external.
1.6 Host Infrastructure
The Host infrastructure is nothing more than you underlying Hypervisor hosting you virtual XenDesktop (VDI) and/or XenApp (HSD) machines. This can be XenServer, Hyper-V or VMware vSphere (ESXi). The host connection is configured in Studio and basically connects your Delivery Controllers to the Hypervisor, linking them together. From there on the Delivery Controller(s) will constantly communicate with the Host Infrastructure and tell it what to do: start up a VM for example, tell a VM that a new connection is coming, create new machines using MCS, log all configuration and/or machine connection state changes, monitor the infrastructure etcโฆ
1.7 Site Database
The central (SQL) Site database. This is where all information, dynamic and static, is stored. It holds all Site configuration information with regards to (HDX) Polices, Delivery Groups, Machine Catalogs etc. Itโs also responsible for storing all dynamic session information and as such it has a constant connection with the Deliver Controller(s). It needs to know everything thatโs going on.
If a Delivery Controller needs to broker a new session or reconnect an old one, this is where he (is it a he?) gets his information. As mentioned version, 7.6 introduces a new feature called Connection Leasing (see the link earlier), if for whatever reason the central Site database becomes unreachable, never mind if itโs HA or not, users will still be able to login and use all their resources that they have accessed at least once during the past two weeks (default setting) as of that moment.
1.8 Citrix Studio
Citrix Studio, the main management console from where you configure and manage the biggest part of your XenDesktop Site deployment. Although there are several options and/or features that you can only configure using PowerShell, Studio has a build-in PowerShell prompt as well. However, if you use PVS for example, thatโs still a separate management console, the same applies to Director and StoreFront. The latter can be added to Studio as well but is only partly manageable. NetScaler is also separate, but then again, it isnโt really part of the FMA, I guess it all depends on your point of view.
1.9 Citrix Director
I already briefly mentioned Director in the previous paragraph, it is Citrixโs first line of defence monitoring and troubleshooting tool. It comes shipped with XenDesktop/XenApp. As of XenDesktop 7.x it has some of the former EdgeSight functionality build-in and if you own the proper licenses (XenDesktop and NetScaler) you can monitor your NetScalers (HDX insight) using the same console as well, again, based on EdgeSight technology. It offers some really nice overviews and statistics.
You can filter on failed machines and sessions, drill down deeper on a per user basis, see which applications are in use, shadow sessions, kill process and/or applications, log off sessions, reboot machines etc. On the main dashboard (in the administrative view that is) it also clearly displays the average login time and the time it takes per process during login, like GPO processing for example. It also monitors your Delivery Controllers on an ongoing basis using PowerShell under the hood.
1.1.1 ICA / HDX stack
There is a lot I could write about the ICA protocol, including HDX for that matter, but for now letโs just say that without the ICA protocol stack it wouldnโt be half as sufficient and smooth as it is this today. It is of course THE protocol used to communicate from client (Receiver) to virtual or physical machine (VDA), internally as well as externally.
Ok, thatโs it with regards to the FMA. Letโs get to the fun stuff.
2. User authentication, internally and externally.
User authentication takes place internally, through StoreFront or Web-Interface, or externally through NetScaler (something that also needs to be configured on the StoreFront or Web-Interface server) which will than pass on the user credentials up to either StoreFront or Web-Interface.
2.1 StoreFront vs. Web-Interface
Before we go into the user authentication and application enumeration processes, a few notes on StoreFront and Web Interface first. Both products offer a different feature set, which Iโm not going to fully discuss here, and handle certain functionality in different ways. One of which is how they take care of user authentication as part of the resource enumeration process, something that will automatically happen when someone starts a new session. Note that we are only referring to XenDesktop 7.x Sites here since StoreFront is also able to authenticate users to, and enumerate and aggregate resources from, XenApp Farms, XenMobile App Controllers or VIAB for example.
2.2 StoreFront
When using StoreFront users are authenticated by the authentication service, which is an integral part of StoreFront. Users can authenticate to StoreFront using different methods: using usernames and passwords for example or Domain pass-through, NetScaler pass-through, using Smart cards or by enabling unauthenticated user access. As soon as a user logs in by filing in his or her username and password (on the StoreFront web page using the so called Receiver for web sites configuration, or using a locally installed Citrix Receiver) the StoreFront authentication service will pick up the user credentials and authenticates them with a domain controller.
Once authenticated StoreFront will forward the user credentials, as part of a XML query, to one of the configured Delivery Controllers. Assuming you configured at least two of course. In between StoreFront will check its local data store for any existing users subscriptions and stores them in memory. Next, the Delivery Controller receiving the credentials will again contact a domain controller this time to validate the users credentials before responding back to the StoreFront server.
Note how I say user authentication and user validation! There is a distinct difference. Authentication is to make sure that somebody is who he or she claims to be. Verification is done to find out which resources are assigned (permissions) to the user, which will then be displayed in the users store, ready for subscription.
2.3 Web-Interface
If we look at Web-Interface, user authentication works a bit different, it has no internal authentication service. When a user logs in by filling in his or her username and password, Web-Interface will immediately forward these credentials, as part of a XML query, to a Delivery Controller where a domain controller will be contacted to authenticate the user before responding back to StoreFront. As like before, Web-Interface can also authenticate users to, and enumerate and aggregate resources from, multiple XenApp Farms and XenDesktop Sites, no App-Controller though.
So there you have it, similar solutions, same result, just another route. Iโm aware that the Web-Interface XenDesktop 7.x combo probably wonโt be around much longer (Web-Interface will end of life in 2015) and even if it will, most newly build XD 7.x deployments are build using StoreFront anyway, so I doubt if there are that many to begin with. I just wanted to make clear the difference in authentication since this tends to confuse people from time to time.
3. Resource enumeration
When a user logs in for the first time, meaning that there are no active and/or disconnected sessions lingering around somewhere, right after the user is authenticated the resource enumeration process kicks in and will eventually show the user its assigned resources. Thatโs why the user authentication process and resource enumeration basically go hand in hand. Letโs put the two together and see what happens, Iโll only use StoreFront in my example.
3.1 Applications subscription
Due note that when using StoreFront users will first have to select and subscribe to applications from the store before they show up on their main home screen and can be launched. When using so called Keywords, Administrators can pre-subscribe users to certain core applications so that their home screen wonโt be completely empty when logging in for the first time. This also works for assigned Desktops. Use Keywords:auto with the application and/or desktop of choice.
3.2 External user authentication through NetScaler
Letโs take it step by step and see what happens when someone logs in externally. Iโll assume that your NetScaler Gateway is set up and configured to integrate your StoreFront server(s), you have Receiver installed, SSL certificates are present, and that a STA / XML / Broker service address (Delivery Controller) and a domain controller for authentication purposes are also configured. Perhaps you want to load balance your StoreFront and/or Delivery Controllers by creating and configuring virtual load balance servers on the NetScaler, adjust the theme of the NetScalers Web Interface and last but not least, youโll probably have to configure your StoreFront deployment to accept pass-through authentication from NetScaler. Where port Nr. 443 (SSL) is used you can also use port 80, although 443 is recommended.
- *A user opens up a web browser and connects to the external URL of the NetScaler Gateway (using SSL over port Nr. 443) here he or she will fill in his or her username and password. A locally installed Citrix Receiver can also be used to establish a direct connection to the NetScaler Gateway. Citrix Receiver uses so called Beacons to determine if a connection is internal or external and handles it accordingly. Check out the link.
- The NetScaler will take the user credentials and authenticate them (session ticket) against Active Directory over TCP port Nr. 389. The NetScaler has its own authentication service just like StoreFront mentioned earlier.
- Once authenticated the user session gets redirected to StoreFront where it will first perform a callback to the NetScaler that handled authentication to validate the user. The authentication details will then be send to the StoreFront.
- From here the user credentials will be forwarded, as part of the earlier mentioned XML query, to the configured Broker (XML) service on one of the Delivery Controllers. Both these transactions will use port Nr. 443 / SSL.
- In between, StoreFront will check its local data store for any existing recourse subscriptions and stores these in memory.
- The Broker (XML) service will again contact a domain controller (port Nr. 389) to validate the user credentials, note that this is different to the user authentication process, as weโve established earlier. During this process it will find out to which security groups the user belongs.
- With this information the Delivery Controller, or Broker (XML) service, will contact the central Site database to find out which resources have been assigned to the user. It does this over port Nr. 1433 / 1434
- The Broker (XML) service will return an XML file to the StoreFront server including all assigned resources.
- StoreFront will generate a web page containing all the assigned resources, which will be routed through the NetScaler Gateway and presented to the user. The users home screen will be populated with any pre-subscribed resources (Keywords). Depending on how you connected, your resources will be displayed either directly using a Receiver for web sites or youโll find them within the locally installed Citrix Receiver instead. The user will be able to browse its own personal app store for any assigned resources to which he or she can subscribe and then launch.
*If you donโt enable authentication on the NetScalers login page the NetScaler will contact StoreFront and the user will be presented (through the NetScaler) with the StoreFront login page (Receiver for web sites). The user fills in his or her credentials and authentication will be handled by StoreFront.
Make sure to also check out my latest article on the Citrix NetScaler: Citrix NetScaler… The basics continued, part one. VIP’s, Monitors and other object! More will follow.
3.3 Internal user authentication through StoreFront
So what happens when a user authenticates internally, directly to StoreFront? Letโs have a look. Same rules apply here, use port Nr. 443 where you can.
- A user opens up a web browser and connects to the internal StoreFront URL where he or she will fill in his or her username and password. This method is also referred to as Receiver for web sites as mentioned above (donโt confuse this with the HTML 5 based Receiver for web, theyโre not the same). A locally installed Citrix Receiver can also be used to establish a direct connection to StoreFront, which is probably the preferred method whenever possible. The earlier mentioned (NetScaler) Beacon functionality applies here as well.
- Next the StoreFront authentication service will pick up the user credentials and contact a domain controller to authenticate the user in Active Directory over TCP port Nr. 389. Here Iโd like to note that if domain pass-through authentication is enabled on the StoreFront server, this step would automatically be skipped.
- Once authenticated the user credentials, as part of the XML query, will be send to a Delivery Controller.
- In between, StoreFront will check its local data store for any existing recourse subscriptions and stores these in memory.
- During the next phase the Broker (XML) service will again contact a domain controller (port Nr. 389) to validate the user credentials, this is different to the user authentication process, as weโve established earlier. During this process it will find out to which security groups the user belongs.
- With this information the Delivery Controller, or Broker (XML) service, will contact the central Site Database to find out which resources have been assigned to the user. It does this over port Nr. 1433 / 1434.
- The Broker (XML) service will return an XML file to the StoreFront server over port Nr. 443 / SSL.
- StoreFront will generate a web page containing all the assigned resources, which will be presented to the user. The users home screen will be populated with any pre-subscribed resources (Keywords). Depending on how you connected, your resources will be displayed either directly using a Receiver for web sites or youโll find them within the locally installed Citrix Receiver instead. The user will be able to browse its own personal app store for any assigned resources to which he or she can subscribe and then launch.
4. The launch process
Here we basically pick it up were we left off at the end of the resource enumeration process as explained above. Just as with the authentication process, there are some differences between the internal and external resource launch process. Also, when launching a HSD or a published application there is an extra load balance step involved as well. Letโs start with an external HSD launch trough NetScaler. Note that the below process is basically XenApp as we knew it earlier.
4.1 The Secure Ticket Authority
Before we continue… You might have heard about something called the STA, or the Secure Ticket Authority in full. It was first introduced with one of the earlier Secure Gateway editions over ten years ago. It (the STA) runs as a service and is part of the Broker service on the Delivery Controller just like the XML service. During the resource launch sequence the StoreFront server as well as the NetScaler will both need to be able to communicate with the STA. As such you will need to configure the NetScaler as well as the StoreFront server(s) or Web-Interface server(s) to point to the same XML/STA service(s)/Deliver Controller(s).
Once a user launches a resource, externally through NetScaler Gateway, at one point a secure ticket will be requested. As we will see shortly the STA ticket will eventually end up in the launch.ica file generated by StoreFront and/or Web-Interface. Once generated, the Delivery Controller hosting the STA service will hold the STA ticket information in memory for a configurable amount of time. As soon as a secure session is established the NetScaler Gateway responsible for handling the session only has to check the STA ticket (as part of the .ica launch file) with the STA service that originally generated the ticket. It (the STA service) does this from memory where the ticket was stored after it was created and send back to the StoreFront server as part of the XML file mentioned earlier. More (detail) on this in the overview below.
The STA is only used when traffic traverses a NetScaler, so you don’t have to worry about the STA service and its tickets when authentication takes place internally.
- Assuming that the login and enumeration process finished without any issues (see above) the user is free to subscribe to and launch any applications and/or desktops that have been assigned to him or her. As an example, letโs say that the user tries to launch a (XenApp) Hosted Shared Desktop session.
- After the user clicks the icon the launch request is send to the NetScaler Gateway from where it will be forwarded to the StoreFront server.
- The StoreFront server will contact the Broker (XML/STA) service, or Delivery Controller, to find out if and where the resource is available and where it can be best started. This is where the well-known XenApp load balancing mechanism comes into play. Which as of FMA needs to be configured through policies.
- During this time the StoreFront server will also request an STA ticket from the Broker (XML/STA) service. It will include the user, domain and resource name it wants to start. It will also request a โleast loadedโ server as part of the load balancing process.
- The Broker (XML/STA) service will query the central Site database (ports Nr. 1433 and 1434) to find out which server is able to offer the requested resource. The Delivery Controller will use this information together with its load balance algorithm to decide which server to connect to.
- At this time the Broker (XML/STA) service will create the STA ticket mentioned earlier. This will include information on the server and resource to connect to, as discovered in the previous steps mentioned.
- Next the Broker (XML/STA) service will send this information back to the StoreFront server in the form of an XML file.
- Based on this information the StoreFront server will then generate a launch.ica file (uses the default.ica file as a template) containing the STA ticket and a whole bunch of other connection properties that are, or might, be needed. This will also include the FQDN/DNS name of the NetScaler Gateway itself.
- StoreFront passes on this information down through the NetScaler Gateway onto the locally installed Receiver, which initiated the connection to begin with.
- *The locally installed Receiver will read and autolaunch the launch.ica file to set up a connection to the NetScaler Gateway (443 / SSL).
- From here the NetScaler Gateway will first contact the Broker (XML/STA) service (this address is configured on the NetScaler as well) to verify if the earlier generated STA ticket, as part of the launch.ica file, is still valid.
- The Broker (STA) service will validate the STA ticket from memory. Once verified it will send back the IP address, port Nr. Resource name etc. of the machine and the resource it needs to connect to. Once done the STA ticket will be deleted.
- The NetScaler Gateway will set up a new ICA connection using port 1494 (ICA) or 2598 (CGP โ Common Gateway Protocol) depending on config.
- The installed VDA will verify its license file with the Delivery Controller.
- The Delivery Controller checks with the Citrix License server to verify that the end user has a valid ticket. This will also be done for any Microsoft (CAL) licenses, with regards to HSD and published applications, that might be involved.
- At this time any applicable session policies will be passed onto the VDA applying them to the session.
- Finally the HSD is launched and the NetScaler Gateway acts as a proxy between the user and the XenDesktop resource in the data center.
- Somewhere in between the session/connection information will be passed on and registered in the central Site Database where it will be used for future load balance purposes.
Thatโs right, the STA ticket gets generated and send back after a user launches an application/desktop, not during the resource enumeration process.
*In my example I assume that you already have the Citrix Receiver installed locally, which is a pretty common scenario. But if you donโt, you have a few options. First, when you connect to StoreFront, either directly or through the NetScaler Gateway as weโve talked about, it automatically checks if there is a Receiver installed locally. If not, in most cases it will guide you to a download section or page (usually Citrixโs) where you will be able to download the Citrix receiver. There are a few ways administrators can implement this.
4.2 HTML 5 to the rescue
If for whatever reason you are unable or not allowed to install a Citrix Receiver locally, Citrix offers the Receiver for HTML 5. You will still be able to connect to StoreFront / NetScaler and launch your resources without any loss of functionality. Although not enabled by default, StoreFront has a build-in HTML 5 based Receiver, which will kick in at launch time. It does this by fetching the HTML 5 engine from the StoreFront and making it part of the local browser. Note that you must use a HTML 5 supported browser for this to work. So basically your browser becomes your Receiver handling the launch.ica file. When you close the browser, you close the session. Even when your users will have Receiver installed you can enable it anyway as it will function as a fallback mechanism.
4.3 Broker, XML and STA.
Be aware that the STA (service) is also part of the Broker service, and has been as of Presentation Server 4.0. Before that it was written as an ISAPI extension for Microsoft Internet Information Services, or IIS. I also highlighted the so-called XML service multiple times. I put the XML and STA services between brackets because as of XenDesktop 4.x the XML service (ctxxmlss.exe) has been rewritten in .NET and became part of the Broker service. So the Broker service is actually build up out of three separate services, all handling different tasks, it brokers connections, it enumerates resources and it acts as the Secure Ticket Authority, generating and validating STA tickets.
Make sure that the Broker (XML/STA) service on the NetScaler and the Storefront server is configured identically. The same applies to the load balance/fail over order in which you configure them.
4.4 Launching an internal resource
Now that we’ve seen which steps are involved when launching a resource externally, a Hosted Shared Desktop in this case, letโs have a look and see what happens when we launch a pooled VDI virtual machine internally. After this we will have looked at an external and internal resource launch, a HSD, which is comparable to a published application, and a VDI virtual machine. Again, user authentication and resource enumeration has successfully completed, here we go (again).
- As mentioned we will launch a pooled VDI virtual machine this time. Letโs assume that the VM is pre-subscribed and already present on the users home screen, never mind how we connected: locally installed Receiver or using the Receiver for web sites.
- After the user clicks the icon the StoreFront server will contact the Broker (XML/STA) service, or Delivery Controller, to check if any registered VDAโs are available. It does this by communicating with underlying Hypervisor platform through the Host service on the Delivery Controller.
- If needed it will first start / boot a VM. Itโs not uncommon to pre-boot a few VMโs, since, as you can probably imagine, this will positively influence the overall user experience.
- Next the Delivery Controller, or Broker (XML/STA) service, will contact one of the VDAโs and sends a startlistening request. By default the VDA isnโt listening for any new connections on port Nr. 1494 or 2595 until it gets notified that a user wants to connect.
- As soon as the VDA is listening, the Broker (XML/STA) service will send this information back to the StoreFront server in the form of an XML file.
- Based on this information the StoreFront server will then generate a launch.ica file (it uses the default.ica file as a template) containing the IP address of the VDA and a whole bunch of other connection properties that are, or might, be needed. This is send down to the user.
- The locally installed Receiver (or HTML 5 based Receiver) will read and autolaunch the launch.ica file initiating a direct connection from the users end-point to the VDA.
- The installed VDA will verify its license file with the Delivery Controller.
- The Delivery Controller checks with the Citrix License server to verify that the end user has a valid ticket.
- At this time any applicable session policies will be passed on to the VDA and the session is launched.
5. What happens inside the VDA during launch time?
Letโs take it one step further and see what happens inside the VDA during launch time. The process below assumes that session reliability is enabled and that a desktop OS VDA gets launched as weโve seen in the previous section, the below takes place somewhere around step 7. Remember that as of XenDesktop 7 there is also a server OS based VDA.
- The CGP service will receive the connection and sends this information on to the tcpip.sys, which will forward it to the ICA stack.
- The ICA stack will notify the ICA service a.k.a. the Portica service (Picasvc) that a connection has been made after which the Picasvc will accept the connection.
- Then the ICA service will lock the workstation because the user needs to be authenticated to ensure that the user is allowed access to that particular machine.
- As soon as the user logs on to the workstation the Portica service will communicate with the display manager to change the display mode to remote ICA, this request will be forwarded to the Thinwire driver.
- In the meantime the Portica service will hand over the โpre logonโ ticket data, which it received from the ICA stack, up to the Desktop service and from there back to the Delivery Controller in exchange for โrealโ credentials.
- The Desktop service receives the users credentials, which are send back to the Portica service.
- The Portica service contacts the authentication service to logon the user and this is sort of where the process ends.
Itโs kind of hard to find information on these kinds of traffic flows. I was fortunate enough to some across an older Citrix Synergy presentation, which handled the subject in great detail, it helped me a lot. Do note that with the newer XenDesktop 7.x versions things might have slightly changed, but I still think this should give you a good idea on what’s going on internally. It was presented by Karen Sciberras by the way a.k.a. @XMTipster on Twitter, currently sheโs deep into XenMobile.
5.1 So what about the server OS VDA?
UPDATED 16-12-2015 – I worte a new article on the Server VDA, it includes more details on the inner workings.
Well, there is not that much to tell to be honest, information is scares. There are a few (older) Citrix blogs which mention the new server OS VDA and how itโs supposed to be kind of comparable to the former XenApp 6.5 worker only role (remember that one? It couldnโt be elected as a Data Collector) only way more efficient. It has been built from the ground up and it should have a lot of similarities with its little brother, the desktop OS VDA. Or is it big brother? It is a lot older.
Anyway, hereโs what they wrote about it when they were on the verge of releasing XenDesktop 7, we have this:
- The Excalibur Delivery Agent for Windows Server Machines is designed from the ground up for dynamic provisioning with Machine Creation Services and Citrix Provisioning Services. Unlike XenApp, the Delivery Agent communicates only with the controllers in the site and does not need to access the site database or license server directly.
- Delivery Agents do not need to run the same version or OS as the controllers. This simplifies the process of upgrading sites, and allows Excalibur to support six different Windows operating systems all within a single farm: Server 2008 R2, Server 2012, XP, Vista, 7, and 8.
- While the XenApp session host-only mode disabled functionality and offered performance benefits over XenApp controllers, it consisted of the same installed services and same binaries as the controllers. The Excalibur Delivery Agent for Windows Server Machines is lighter weight, and only consists of the components needed for hosting sessions. It does not share any common installed components with the controllers.
And then we have:
The biggest difference between the two Delivery Agents is the ICA protocol stack. For desktop machines, Citrix ships a single-user ICA stack (internally known as Portica) which allows only one ICA session at a time. This version connects users to the machineโs console session, similar GoToMyPC or other Remote Access products for a Desktop OS. It also includes additional HDX features such as USB and Aero redirection, which are only available on a single-user machine. For server machines, Citrix includes a multi-user ICA stack, which extends Windows Remote Desktop Services with the HDX protocol. This is the same ICA protocol stack developed for Citrix XenApp, just with a different management interface to make it compatible with Excalibur controllers.
Which also sounds familiar, although Iโm not too sure on the server OS VDA ICA stack though. Like I mentioned before I couldnโt find any real useful information on the internals with regards to the server OS VDA. Butโฆ I got a lot of help from Mick Glover a.k.a. @XDTipster on Twitter (thank you Mick) so I wouldnโt be surprised if I can update this article with some new and interesting facts not to long from now. Stay tuned.
6. The ICA protocol internals.
I thought hard on what to do with this one, I mean, there is just so much to still learn and tell with regards to the ICA protocol and all that surrounds it, I wouldnโt even know where to start. But then I came across this presentation given by Dennis Gundarev, a.k.a. @fdwl on Twitter, at BriForm this year. Thatโs it, it will tell you all you need to know and more, make sure to give it a look.
Since a great deal of the communication flow within the ICA protocol, like graphics, printers, audio, smart cards and what not, takes place through virtual channels (inside ICA) I thought Iโd throw this one in as well.
7. A (deeper) look into XenDesktop 7.x
Have you ever wondered what it is that keeps XenDesktop spinning once weโve installed our first Delivery Controller and Studio along side it? Which service is responsible for interacting with our virtual machines, how new machines are created, how delegation of control is managed and a few more of those day-to-day tasks that we normally take for granted? During the next few sections Iโd like to highlight a few of the lesser-known concepts around Citrix XenDesktop.
I think we should just call it FMA from now on and use the technology stacks, XenDesktop (VDI) and XenApp (HSD), to indicate what we offer to our users and how we deliver it.
The Delivery Controller as part of XenDesktop is the hart of the FMA. It communicates with Studio, Director, the central Site database and the VDAโs installed on the virtual and physical machines in our datacenter. Once installed it is build up out of ten primary services that take care of almost everything from there on. They communicate with the underlying Hypervisor and the virtual machines on there, the creation of new machines, delegated administration, configuration logging, service interaction and much more. Here they are:
- Broker service โ Probably the bet known one. It brokers new sessions, handles resource enumeration and the creation and verification of STA tickets.
- Configuration service โ Although most people probably donโt know it, this is probably one of the most important services in there. It makes sure that all services are able to communicate with each other.
- ID Identity service โ It handles all Active Directory computer accounts related to XenDesktop virtual and physical machines.
- Configuration Logging service โ It monitors and logs all configuration changes made within a XenDesktop Site, including all Administrator activity.
- Delegated Administration service โ This manages the configuration of all delegated administrative permissions.
- Machine Creation service โ Handles the creation of new virtual (not physical) machines.
- Host service โ Manages all connections between the hosts and the underlying Hypervisor.
- Environment Test service โ Manages tests within your XenDesktop Site, can be initiated from Studio for example.
- Monitor service โ Monitors the overall FMA architecture and produces alerts and warnings when it finds something is potentially wrong.
- StoreFront service โ Manages your StoreFront deployment.
Now that you know the services that make up a XenDesktop Delivery Controller, you can probably imagine how all these services will, and need to, interact to get keep things going. This is one of the main reasons why the Configuration service is such an important one, it handles all inter service communication.
All other FMA/XenDesktop services need to register with the configuration service on startup so that it knows they are all good to go. Or in the words of Mr. XDTipster: It is the glue that sticks FMA together!
7.1 They are all independent
All FMA services run completely independent from each other; if one goes offline it wonโt directly affect any of the other services. Although each service points to the central Site database, they all have an independent location in the registry, meaning that their connection strings to the database are all stored separately from each other, eliminating any single point of failures.
All FMA services run under the NT AUTHORITY\Network service account. Also, when authenticating to the central Site database, all service use the computer account that they are running on.
7.2 XenDesktop (Site policies) using PowerShell
The states of your FMA services are best checked using PowerShell. Using some of the PowerShell Get- commendlets when checking up on your FMA services will show you exactly what is going on, when and if something is wrong. Itโs much more detailed then using the services.msc window for example.
If you have a central management server I suggest you create a personal PowerShell profile and include some of the basic Get- FMA service checks in it. This way, every time you open PowerShell these basic checks will be done automatically before you continue. If you look at Director for example, on the main dashboard, there you also see your Delivery Controllers listed at the bottom of the screen. If all is well, green checkmarks pop up next to them. This is also PowerShell issuing Get- commands in the background.
Here are a few examples to check some of the more important FMA services:
- Get-BrokerServiceStatus
- Get-ConfigServiceStatus
- Get-HypServiceStatus
- Get-AcctServiceStatus
- Get-ProvServiceStatus
Another thing to mention is that both Studio and Director run on top of the PowerShell SDK as well. Everything you can do within Studio can also be done through PowerShell, including a whole bunch of configuration options and tweaks that are not possible using โjustโ Studio.
When you check your Delivery Controllers in Studio, youโll see a number in minutes next to each Controller that indicates when the Delivery Controller has last registered itself with the central Site database. This number should always be 0. By default the Controller checks in every 20 seconds, which will then be valid for another 40 seconds.
7.3 Site policies
When we publish resources, either Hosted Shared Desktops, VDI based virtual machines or just some applications on their own, we normally would use a combination of Catalogs and Delivery Groups to grant or allow access to these resources. Although this works fine in most cases, using PowerShell we can get a bit more granular.
Entitlement policies โ These apply to pooled and shared desktops. With entitlement policies you can explicitly deny a certain user from a group of users access to a pooled and/or shared desktop. So you have a group of letโs 50 users and you want to exclude five users, using these policy rules you wonโt have to create a separate group of users to exclude, you can just exclude those five users without affecting any of the other users. There are two Entitlement polices:
- BrokerEntitlementPolicyRule, this one issued for access to desktops.
- BrokerAppEntitlementPolicyRule, this one is used to control access to applications.
Assignment policies โ These basically do the same thing as the Entitlement policies described above only they apply to dedicated private desktops. Again, there are two policies:
- BrokerAssignmentPolicyRule, this one issued for access to desktops.
- BrokerAppAsignmentPolicyRule, this one is used to control access to applications.
Before we move on to the Site Access policy Iโd like to point out another Entitlement policy gotcha:
Once weโve configured a Delivery group with the Desktops and Applications delivery type we can use PowerShell to limit access to the HSD. Let me explain what I mean here. By default, when you create a delivery group with the delivery type set to Desktops and Applications, Studio creates one Entitlement Policy Rule and one App Entitlement Policy Rule for the group, meaning that each user is entitled to one desktop session and one app session. Studio doesnโt expose the user filter on these objects, so both are available to all users of the delivery group.
Using the PowerShell command: Set-BrokerEntitlementPolicyRule we can change this behavior. It can set the IncludeUserFilterEnabled parameter to True instead of False, enabling the user filter, and it also lets you add in an AD security group, this way limiting access to just that group and that group alone as apposed to all users that are a member of the Delivery group.
7.4 Site Access policies
Site Access policies โ This isnโt directly about the users connecting, it is more about connections in general and the conditions that need to be met once a connection gets established, things like client IP addresses, the protocol used, Smart Access filters, host names etc. Based on this information connections can be excluded or denied access as well.
When a Delivery Group gets created two access rules are created and added by default, one for direct connections and one for connections through NetScaler. Using PowerShell we can look at and change these access rules, as we feel fit. To see what your Site Access policies currently look like, open PowerShell and type:
Get-BrokerAccessPolicyRule followed by the โDesktopGroupName command, so that you wonโt get overloaded with all Access policies currently enabled. To edit these policies you will use the Set-BrokerAccesssPolicyRule command just like you would with the earlier mentioned by the way.
8. Key takeaways
- Within a XenDesktop 7.x Site there are basically two points of authentication, the StoreFront server (internally) and the NetScaler Gateway (externally).
- The STA is only used when traffic traverses a NetScaler, so you wonโt have to worry about the STA service and its secure tickets when authentication takes place internally.
- StoreFront directly authenticates the user with a Domain Controller. Web Interface passes on the user credentials to the Delivery Controller where it will authenticate the user against Active Directory.
- User authentication and user validation! There is a distinct difference. Authentication is to make sure that somebody is who he or she claims to be. Verification is done to find out which resources are assigned (permissions) to the user, which will then be displayed in the users store, ready for subscription.
- Do not confuse the HTML 5 based Receiver with the Receiver for web sites configuration; theyโre not the same!
- Once authenticated and resource enumeration is completed the users home screen wonโt be automatically populated with resources unless Keywords (on the published resource properties) are used.
- If you donโt enable authentication on the NetScalers login page the NetScaler will contact StoreFront and the user will be presented (through the NetScaler) with the StoreFront login page. The user fills in his or her credentials and authentication will be handled by StoreFront.
- The STA ticket gets generated and send back only after a user launches an application/desktop, not during the resource enumeration process.
- Both the STA and XML service are part of the Delivery Controllers Broker service. So the Broker service is actually build up out of three separate services, all handling different tasks, it brokers connections, it enumerates resources and it acts as the Secure Ticket Authority, generating and validating STA tickets.
- Make sure that the Broker (XML/STA) service on the NetScaler and the Storefront server is configured identically. The same applies to the load balance/fail over order in which you configure them.
- When connecting externally through a NetScaler Gateway, the launch.ica will include the FQDN / DNS name of the NetScaler.
- The Broker (STA) service will validate the STA ticket from memory. Once verified it will send back the IP address, port Nr. Resource name etc. of the machine and the resource it needs to connect to and start. It gets deleted right after.
- XenDesktop is made up out of ten primary services in total; the Configuration service is probably the most important service of them all.
- The Configuration service makes sure that all other services can communicate with each other. A service broker if you will.
- All other XenDesktop service need to register themselves with the Configuration service at startup.
- All FMA services run completely independent from each other, if one goes offline it wonโt directly interfere with one of the other services. Each service points to the central Site database, however, they all have an independent location in the registry, meaning that their connection strings to the database are all located separately from each other, eliminating any single point of failures.
- Make sure you know about the Site Entitlement and Assignment policies, including the Site Access policies and how to view and edit them using PowerShell.
- Check your FMA service using PowerShell; try to automate this process.
9. Round up
That about rounds it up for now, I hope you found it somewhat informative and if you have any additions or other remarks do let me know. As always I loved putting this together and it taught me a thing or two as well. Again a big thank you to Mr. @XDTipster for backing me up on some of the content discussed.
41 responses to “The ultimate Citrix XenDesktop 7.x internals cheat sheet!”
How may I get a password to this please??
It will come online within a few hours :-)
Awesome Article, thanks !! :)
Thanks a lot.
Hey Bas
Once again excellent article and very well explained . Thank you for spending time to write such a huge article :) . I have been reading knowledge base articles in many blogs and the difference between other blogs and yours is that the you have the ability to explain the basics very well and you are willing to share the knowledge .Keep up the good work :):)
Thank you very much! Glad it helped.
Bas,
Thank you for the extremely comprehensive article. Really phenomenal!!
One question:
How is the user actually logged in to the XenApp Session Host (after the STA ticket is resolved to find out where the session host is). Are the user’s credentials in the ica file created by StoreFront and the NetScaler passes them to the VDA? If so, are they encrypted within the ICA file (like the IP address of the session host)? I’ve heard speak of a logon ticket that the Delivery Controller creates, storing the credentials in memory, and then the logon ticket is stored in the ica file and exchanged with the delivery controller to perform the login into the session host (which is similar to how the STA service works) but I was hoping you could specify/elaborate…
Thanks,
Paolo
Hi Paolo, first of all, thanks for the kind words.
Yes, they are part of the launch.ica file (ticket), and no, they are not encrypted by default. At least not when traffic is internal a.k.a. backend traffic. You will need to come up with a secure delivery mechanism yourself, like secure ICA and / or SSL for example. Have a look at the links below, I think they will answer all your questions:
http://blogs.citrix.com/2014/10/16/xenapp-and-xendesktop-7-6-security-fips-140-2-and-ssl-to-vda/
http://www.ingmarverheij.com/the-citrix-ica-file-explained-and-demystified/
http://www.slideshare.net/fdwl/citrix-internals-ica-connectivity
Hope this helps!
Regards,
Bas.
Thank you!
Sure thing. Enjoy your weekend.
Hi Bas,
you write that the VDA can be supported on Windows XP – according to Citrix official eDocs this is not the case.
Have you actually tried to install the VDA for XenDesktop 7.6 on a Windows XP? I would like to know if it is possible as I want to publish VM Hosted apps.
Thanks.
Hi, No, that’s not what I wrote. That is a quote from one of the older Citrix blogs. I would do what the eDocs tell you to do :-)
We currently use VDIs in XenDesktop 6.5 environment and are told by Citrix Consulting that if WinXP is necessary to keep on some VDIs, we should be able to continue to use the installed older XD5.6 VDAs on these VDIs (which are compatible with Windows XP) to connect to the new XenDesktop 7.6 system when it has been built. Of course it’s a temporary solution while the last XP VDIs are phased out.
Sorry, missed this on earlier, thanks for sharing.
Very nicely done, Bas. I know all this stuff, but I had a wrong understanding of StoreFront authentication. I’ve set it up for over a dozen customers, but I had been assuming that authentication was occurring the same way as it did with Web Interface (through an XML service request to the XA/XD Controller). But you are right: authentication is actually happening on StoreFront, or at the NetScaler. That should have been self-evident, because SF has to be a domain member and the Citrix Credential Wallet service is integral to the operation of SF, but I didn’t really think about it until I read what you had to say. I enjoyed your very thorough presentation. Thanks for sharing!
No, thank you hans :)
[…] https://www.basvankaam.com/2014/11/24/the-ultimate-xendesktop-7-x-internals-cheat-sheet/ […]
thanks for sharing storefrontvs webointerface easy understandable. quick question we have internal users coming via netscaler-> firewall-> sf, backend servers. here you mean to say this users are belongs to internal.
Hi Sentil, sorry for the late reply. I would say yes, but it’s probably best to give these two a quick read: http://discussions.citrix.com/topic/351005-should-internal-user-be-pointing-to-netscaler-or-storefront/ and http://www.jasonsamuel.com/2015/02/17/using-citrix-netscaler-gateway-internally-and-externally-with-a-single-url/
Thanks for sharing
Thanks for reading :)
This is just awesome. I read so many blogs but the way you tear up the internals. just loved it :-)
That’s great to hear, thanks a lot! Glad you liked it.
Hi there,
It will depend how many available configuration service instances you have in your site. You need a functioning service. Its essential. All service instances query it at startup and any time they need to interact. Studio won’t load without working service either. Not a 100% sure what happens to any existing connections though. Hope this helps.
Regards,
Bas.
Thanks alot Bas.. That Helps..
Hi,
Yes that’s possible, from the NetScaler as well as from your Delivery Controllers (XML). Authentication and verification use the same ports. Hope this helps.
Regards,
Bas.
Hi Berks,
Simply put: StoreFront will contact the NetScaler authentication service to verify that the request received from the NetScaler Gateway did actually originate from that appliance.
I’m right :) The STA ticket also includes information on the resource to be launched including the server to launch the application on (load balance).
Regards,
Bas.
Bas, you are a legend and reading your blog has become a regular activity these days. Really appreciate all the effort you put in mate! Thanks again!
Thank you very much!
I’m not sure if it’s also configurable directly from StoreFront. Although the authentication mechanism changed with StoreFront, as apposed to Web Interface, with version 3.x you can also configure it to use XML for authentication like we could with Web Interface, bypassing the authentication service so to speak.
Hello Bas,
You really helped me in clearing all the doubts and questions I had regarding working of xendesktop. Do you have any such articles on PVS as well , the topic seems to be a little complex and an internal architecture might be helpful for clearing all the doubts.
Thanks. No I haven’t sorry.
Great article. Can you only connect to a Hypervisor (our case vmware), during the creation of a new site or can a site be modified to connect to hypervisor after if needed.
Thanks! No, this can be done after the initial Site configuration as well. You can add in a Host Connection any time you would like. Remember, Host Connections are not limited to Hypervisors, they include several Cloud platforms as well.
Regards,
Bas.
Thank You Bas!
No problem! A quick note: Keep an eye out on basvankaam.com I’m working on a book project, which will include this type of information (and a lot more of course) as well. Again, thanks for reading.
Yet another masterpiece :) Thanks for the great write up.
One quick question on the load evaluators in FMA architecture. “Now that with FMA architecture everything is so much dependent on the Data Store, how does the delivery controller/broker server decide the least load server in the event of Data store failure? I know the Connection leasing feature will keep a track of Catalogs, users delivery group, HDX policies including load balancing etc. But how does the load evaluator (as this is a dynamic info) work in the event of DS failure?”
Hi there, unfortunately there is no load balancing taking place when the central site database is down / unreachable. This is on of the drawbacks when compared to LHC for example, which we had in 6.5 and earlier versions (IMA).
Keep an eye out on basvankaam.com I’m working on a book project, which will include this type of information (and a lot more of course) as well. Again, thanks for reading.
Just amazing content. TY for sharing it.
Thank you very much, have you seen my book? Contains information like this and a (whole) lot more :-)
Authority Link Network Ultimate Membership
[…] the STA ticket information in memory for a configurable amount of time. As soon […]