About two weeks ago I had the pleasure to sit down with Dennis Smith – founder and owner of Gourami software. Dennis and I have known each other for a couple of years now and I’ve come to know Dennis as real IT nerd with a big passion for developing software – free (community) software, as well as various ‘paid for’ solutions. Today I would like to introduce you to VDI Drones, a new product released by Gourami aimed at benchmarking/load testing your virtual and underlying physical infrastructure.
UPDATE 05-09-2018
If you haven’t read the article, please scroll down a few sections (to the ‘Introduction’) and return to this point afterward.
First and foremost, the amount of interested has been overwhelming, thank you for that. Multiple use-cases and tons of people wanting to give VDI Drones a spin. There were also some questions about its usability and general ‘fit’ in today’s market, as you can see in the tweet from Andrew below.
Before I continue, I’d like to clear up one or two misconceptions. I’m not the developer of VDI Drones, I haven’t been involved from a technical point of view (at all) up till two weeks ago. Dennis Smith, owner, and founder of Gourami software has developed and launched VDI Drones.
I’m involved from a community perspective, helping to spread the word a little, but that’s it. I do not get anything out of this, though I can imagine this might be hard to believe for some. Since people started to congratulate me, I thought this might worth mentioning :)
Going forward, I can imagine that some of you will have specific technical questions, which is understandable. If I can answer them, I will. But again, it is Dennis (and his company) who wrote the software and who is going to provide the software, the licenses and eventually support when and where needed. But don’t worry, I’m more than happy to function as the intermediate for the time being.
As for some of the questions/remarks made by Andrew earlier, here’s what Dennis has to say about that – I contacted him yesterday afternoon on his holiday address, we probably should have planned this better, haha. And I quote, or for the most part anyway…
Some questions answerd
Hey Andrew, good honest questions, this proves you spend time reading the article and directly point out the right questions IMHO. Thank you for that.
The elephant in the room
I sense that you try to compare VDI Drones with the ‘other big tool’, and I think we all know which one, let’s address the elephant in the room; Login VSI (of course, there are a bunch of others as well). VDI Drones is not, and does not want to be the same as Login VSI. VDI Drones has and follows its own course. The idea behind VDI Drones is to easily generate a synthetic workload on a lot of computers at the same time – keeping it simple and easy to operate. I think the idea behind Login VSI is different.
Throughout the years, while working as a Citrix architect I stumbled upon the same question over and over again; how do we properly size the underlying hardware, and would that be enough to hold a load of an X amount of users.
For example, one time we had to deliver worker pod’s with 10k VDI based VM’s, and we sized 6 PVS servers for it. The aim was to start a load on all 10k machines and see how the PVS servers would hold. Getting 10k user accounts for load testing was out of the question. Within this context, the idea of VDI Drones was born.
The name ‘VDI Drones’ does imply it’s specifically for VDI environment ’s, but it’s not. It can run on any windows based device, physical, virtual etc. supporting all major hypervisors out there. It evolved during load tests performed on a VDI environment, so that’s how we came up with its name.
Question/remark: This does not log users in
Answer: No it does not. We did not intend to make something which does that. We believe that eventually, every login is also a claim om CPU, Memory, IO, and network resources. We tried to avoid creating user accounts, also for political reasons. Often, creating a lot of users in large environments is not allowed, or can take a (very) long time to accomplish. As for pragmatic reasons, to skip creating the users makes testing a lot easier and quicker. Also, the infrastructure required for testing is greatly simplified.
Question/remark: This does not open apps
Answer: No it does not. We are working on a future feature which will allow you to run your own commands (roadmap). But we do not believe this will avoid every problem with a company’s application(s), simply because it does not ever exactly copy user behavior. I’ve seen users crash the most perfectly created apps, no load test tool can avoid this. There is no load test tool that could simulate the most creative, or stupid behavior of some users.
Question/remark: This does not mimic a workload
Answer: In my opinion it does. It’s up to the person who creates the load profile to have an idea of what the actual real-life (work) load would be. Of course, other tooling can help with this as well. It does mimic a workload, however, I cannot guarantee that the load generated exactly matches that of a real user workload. No tool on earth can do this. There is only one real way to do that, and that is to test it with real users…
Question/remark: Does it measure usability?
Answer: It can determine if certain elements of the infrastructure are able to handle a certain (work) load. This all depends on selecting the right load profile, again, this is primarily up to the person who performs the load test, and thus generates the load profiles.
Question/remark: Does it measure users experience?
Answer: No it does not. This is a very tricky thing, it is difficult to quantify, greatly depends on the users’ (or companies’) expectations and is also very relative to how spoiled users are. For example, I’ve seen environments where users complain when the login time was above 35 seconds. I’ve also seen environments where users were happy when the login time was below 3 minutes – and that’s just one aspect/metric of the whole chain.
We are planning to incorporate ‘processor queue length’ in the graph’s since this is an important metric when it comes to the overall user experience. User experience also depends on network latency of your RDP/ICA connection, I’ve never seen tests that actually run simulated ICA sessions from people’s home location or India.
Question/remark: How will this measure changes to the environment?
Answer: You can set a baseline with a certain load profile, tweak/change the environment and re-run the same test.
Hopefully the above has answered some of your questions, though you might still have more, or perhaps disagree with what has been stated here. Both are fine, do let us know. Thank you for your feedback, as I’m sure this will have helped others as well.
One last thing. We are currently in the process of collecting all email addresses of all who want to test/use VDI Drones. However, as highlighted a couple of times already, because we are both still on holiday this will take some time. Thanks again.
Introduction…
Dennis founded Gourami back in 2006 and has since then developed numerous tools. When he started out he quickly noticed that some of his software, Citrix specific back then, was being picked up by a bunch of bigger companies. After polishing up some of his earlier builds, making them market ready, so to speak, he soon saw subscription/download numbers of up 10 to 15 a per day for at least a year and a half. Not bad.
Go here to get an idea of some of the software Dennis developed throughout the years.
After doing tons of virtualization projects, mainly Citrix orientated, but a bunch of VMware and Microsoft ones as well, he has come to know exactly what customers want, need, and what type of software comes in handy every now and again – of course, benchmarking/load testing being one category.
The problem with these types of solutions, in general, is that they can be a hard sell (to management and company executives). Licenses are expensive, they’re usually not straightforward to set up, additional resources are needed, and more often than not a consultant/engineer of the company in question will (need to) join to assist, or to install and configure the lot by him/herself.
Especially for ‘smaller’ environments ranging from 500 to, let’s say 2500 users/machines this can be hard to get across. Of course, the bigger the environment, the more licenses and additional hard and software is needed as well. While discounts will apply, it’s still a shitload of money (and effort), in most cases – pardon my French.
This is where VDI Drones comes in…
An overview
Let’s start with some of its primary use-cases.
- Verify if a newly build infrastructure holds up under load
- Measure the impact of changes, big and small
- For example, to test the impact of patches and updates, like Meltdown and Spectre
- Burn-in testing of new hardware
VDI Drones allows you to benchmark/load test your newly build VDI, Hosted Shared, or DaaS environment – all major Hypervisors are supported. It works for Windows platforms exclusively and is built using Windows Forms as part of the bigger Microsoft .NET Framework, offering a rich graphical user interface – you won’t need any scripting skills to set it up or use it.
Overal, the three main factors that influence how heavy a VM and the underlying hardware is being utilized are CPU, Memory and Disk Inputs and Outputs (I/O) – local and remote. This is exactly what VDI Drones lets you configure. You can specify (in detail), how heavy the load should be in terms of threads, CPU, Memory and read/write I/O consumption (with random idle times in between if you like).
Here’s what you will able to do
- It enables load generation on a large number of machines – tens of thousands of machines at the same time
- It supports Windows (Server and Desktop) 2008, 2012, 2016, 7 and 10
- An easy to use GUI, no scripting skills needed to install, configure and/or operate
- No need to set up hundreds of user accounts – this definitely makes life easier in some environments
- Specify the workload in detail, in terms of threads, CPU, idle times, Memory, and (read/write) Disk IO
- Flawless integration with your Microsoft Active Directory
- Works on all major hypervisors: VMware, Hyper-V, XenServer, Nutanix Acropolis, as well as Microsoft Azure, AWS, etc.
- Competitively priced licenses – very true
- Almost no additional infrastructural components needed
- Available as a portable .exe, no (client-side) installation required
- When the pre-requisites and software are in place you will be able to start load testing on 10.000+ machines in just over 5 minutes. Ok, if you haven’t seen the product before, let’s make it 10 to 15.
Your attention, please…
Do you have a use-case/customer where you would like to use/test VDI Drones? Let me/us know, and we will hook you up with a free unlimited license!!! First come, first serve.
The first 25 people who retweet this announcement, or share it on LinkedIn will receive a free VDI Drones license for testing purposes. Contact me to receive your license.
Note: Near the end of this week/the beginning of next week I will (try and) contact those who have indicated wanting to test, etc. Dennis and I are currently enjoying some time off with our families. Bad timing, perhaps, but we didn’t want to wait with the announcement.
What you will need
Besides a license, there are some prerequisites that will need to be in place, I’ve listed them below. Furthermore, VDI Drones consists out of two main components, the Drone, and the Controller. The Drone generates the actual load. It will be copied over to the VM being tested and removed by the controller when a task is stopped. It is also used to generate/configure a load profile file which will be applied to the VM’s being tested.
The Controller starts stops, etc. the Drones on the remote machines (amongst other things, more on this in a bit) and gathers performance information, which can be viewed live from the Controller.
Other pre-requisites:
- .NET Framework 4.0 or newer
- Active Directory
- 2 GB of RAM as a minimum
- A domain joined computer to run the Controller
- A domain joined computer to run the Drone
- Virtual installation fully supported on Hyper-V, VMWare, AWS, Azure, or Nutanix Acropolis
- Access to the RPC$ and the ability to do WMI queries from the Controller to the Drones (firewall rules and rights)
- Administrative rights on both the Controller and every Drone
Load specification
Since every environment is different, VDI Drones let you specify/configure the load to be applied – see the list below. The generated load profile (and the jobs/subtasks it holds) will be executed/applied in a random sequence. The duration of each subtask is also random within the selected time-frame. As part of a load profile you (can) configure the following subtasks:
- Number of threads
- In between idle time
- CPU in-use percentage
- Memory claim
- Memory copy
- File read
- File write
To give you an example of what a load profile may look like:
Let’s say we configure a single thread (see image below as well). Next, we set the idle time to a minimum of 5 seconds and a maximum of 10. We configure 35% of CPU usage for a minimum of 8 seconds and a maximum of 15. A Memory claim of 50 to 500 MB’s for a duration of 2 to 10 seconds, together with a Memory copy operation of 50 to 500 MB (this will use same time metrics as the Memory claim). And finally, reads and writes. Here you can also configure the file size minimums and maximums, and if storage is local or remote, etc.
All this will be compiled into a single load (.cfg) profile which will be applied to an xxx (for you to decide) number of virtual machines. Since we configured a single thread the configured subtasks will be executed one by one in random sequence.
If we configure two threads multiple subtasks will be executed (two at a time) but again, in a random sequence, on the same machine. You basically configure a single load profile for a specific set of machines. Meaning all machines will get the same load profile.
How to create a load profile
There is a more detailed configuration manual available, but to give an idea of how easy this works, here goes…
Go to the folder where you copied or installed VDI drones. Run the drone.exe software to create a load profile. A standard profile creates a default load on one threat. These settings are saved in a .cfg file, right next to the .exe files.
Next, run the controller software while logged in to the domain with administrative credentials, and select the machines you want to create a load profile on from active directory.
Note that the Controller and the Drone software do not have to be installed. They are portable, just keep in mind to keep them in the same folder at all times. – the controller expects the drone executable and config file in the same folder.
Performance watchers and the Controller
VDI Drones primarily focuses on testing the main VDI/SBC/DaaS infrastructural components, meaning your hypervisor and the underlying hardware it runs on.
During the test, if you, for example, open up vCenter, you will have a perfect view of what is going on, currently and historically. Of course, nowadays the software used to monitor physical and virtual environments greatly varies, just use what you feel comfortable with.
So, what is a performance watcher exactly? You can configure specific key machines/servers (your provisioning servers, Delivery Controllers, certain file servers, domain controllers etc. for example) as a monitoring machine, a.k.a. a performance watcher.
In between, every 15 minutes by default, the collected performance metrics of the performance watcher machines will be saved in an HTML based report. In practice, a single Controller is capable of handling up to 20 watchers as part of 10.000 machines test (verified). Though, when in doubt, I would advise you to talk to Dennis about your (virtual) needs based on your environment – currently, I don’t have any specifics on what a Controller machines should look like from a compute perspective and how it holds up under load.
Personally, I would add in a couple of desktop VMs as performance watchers as well, just as a random sample test. By the way, selecting a machine as a performance watcher is done by checking a checkbox.
If during the test, you log in to your Controller machine you will be able to see a couple of things. For one, everything that happens on your monitoring machines/performance watchers, as mentioned above. The name and number of active Drones, event log entries, and, when you select a Drone you can also see what is happening on that specific machine. The Controller dashboard enables you to install, start, and stop Drones remotely on other VM’s, as well as reset or shutdown VM’s remotely. And before I forget, creating your load profile (s) is also something you do from your Controller machine.
Licensing
Dennis has developed a license tool (digital singed download) for you to run on your environment (run it on the Domain where your Controller and Drones will be deployed) go here for some more information. Based on the outcome you’ll fall into one of 6 categories. After that, just give Dennis a shout to talk about the price. Know that a license file is needed and that once expired it will stop your configured Drones.
Limitations
As with every product, VDI Drones also comes with a few limitations.
- VDI Drones can be used if a certain setup can handle a specific load – if not, at least you’ll now :) This is based on a synthetic load and does not guarantee that an actual user and/or application load can be met. But to be honest, this applies to other solutions as well
- VDI drones does not test or stress network latency for RDP or ICA connections
- The Controller can only start the drones if Windows remoting is allowed, see the manual for more information
- If there is a firewall on the LAN, additional firewall rules, or exceptions may be required
- You need proper (administrative) rights on the target device (Drone)
Concluding
Is it perfect? No. Do you get a shiny polished dashboard with a ton of bells and whistles? Also no, but who cares? Will you get the exact same features as with other solutions (and I know which product you are thinking about now)? Of course not, but that’s not the point either.
You’ll get exactly what you need, otherwise, this solutions isn’t for you. Let me put it this way, you’ll pay roughly 10% of what a competitor license (per seat) costs, but with (at least) up to 50 to 60% of the same functionality/results, probably more in most cases. And besides that, it is extremely easy to set up and implement/configure, and you won’t need any additional infrastructural components or very little at least.
Like any of the other solutions out there, you will not be able to give any 100% guarantees afterward. What you will have is a more than educated guess of what is possible. One of the things I like best though, besides that VDI Drones has already been implemented and used on thousands of production machines (5+ enterprise customers), throughout the last year, is that its roadmap is filled with tons of cool ideas, add-ons, and additional future enhancements.
As a final note, I just want to make clear that I’m involved from a community perspective alone, I have nothing to gain from this. I think it’s a cool initiative, I’ve known Dennis for a while, we got to talk and I decided to help him by means of this article, that’s it.
If you have any questions or feedback, do not hesitate to let Dennis know or leave acomment at the very least.
Thanks for reading.
