I've recently started working on a personal project and one of my goals for this project is to learn how to use Release Management in TFS/VSTS to deploy to an Azure VM. As a QA Test Engineer, naturally, I wanted my project to have automated tests built in. I started creating my project by setting up the project repository in VSO and creating my build. I wrote my first bit of code and associated unit tests. I then committed and pushed my code, expecting everything to go fine because it worked on my local dev environment. I spent several frustrating hours afterward trying to figure out why it didn't work. I want to share the solution in hopes that I can save some hair pulling and head scratching for anyone else trying to accomplish what should be a fairly simple task.

My first mistake was assuming that using Chrome to run my tests would also work in VSO. It doesn't work, and for reasons that should have been obvious. If you look at the installed software for the VSO Hosted Agents here, Chrome is not in the list. No surprise! I should have known that from the start. I knew that I could install PhantomJS locally in my project using npm so I went about doing that.

npm install --save-dev phantomjs-prebuilt karma-phantomjs-launcher

I'm also using gulp and karma to run my unit tests so I added the necessary bits to get the tests running using PhantomJS:

const Server = require('karma').Server;
const path = require('path');
const gulp = require('gulp');

gulp.task("tests", function (done) {
new Server({
configFile: path.join(__dirname, 'karma.conf.js'),
singleRun: true
}, function() { done(); }).start();

And in the karma config:

browsers: ['PhantomJS_custom' ],
customLaunchers: {
'PhantomJS_custom': {
base: 'PhantomJS',
options: {
windowName: 'my-window',
settings: {
webSecurityEnabled: false
flags: ['--load-images=true']
phantomjsLauncher: {
exitOnResourceError: true
plugins: [

I ran the tests locally and everything looked fine so I pushed again, fully expecting the build to pass this time. It failed again. Here is a log snippet from that failed attempt:

INFO [launcher]: Launching browser PhantomJS_custom with unlimited concurrency
INFO [launcher]: Starting browser PhantomJS
WARN [launcher]: PhantomJS have not captured in 60000 ms, killing.

Wait...what? The tests just ran fine on my local machine. Why couldn't it capture the browser? Honestly, I have no idea why this doesn't just work right out of the box. I couldn't find any good explanation but I do know how to fix it. I spent several hours and multiple failed builds trying to get it to work before I finally figured it out. The hardest part was that there really isn't a ton of information online about how to do this so it took a lot of digging around to find the solution.

The first thing you have to do is add a PHANTOMJS_BIN variable to your build definition in VSO and point it at C:\NPM\Modules\PhantomJS.cmd. This worked to get my tests running but then I ran into another problem. The tests executed and passed just fine. However, the build agent stalled on the test build step and I eventually had to cancel the build.


Ok, now what? This doesn't happen locally so there is no way to debug it to see what's wrong. Thankfully, the solution is pretty simple. You just have to tell the gulp task to exit the process when it's done. I updated the gulp task, pushed, and the build FINALLY succeeded. Here's how I changed the gulp task to get it working:

gulp.task("tests", function (done) {
new Server({
configFile: path.join(__dirname, 'karma.conf.js'),
singleRun: true
}, function(exitCode) {
process.exit(exitCode); // <-- this is the important part

Honestly, I spent several hours searching for a solution. I even started down the path of configuring TFS Express in a VM to use as an on-prem build controller. After reading the prerequisites for that task, I decided that I really didn't have the patience to install all the software needed for that. The solution is actually very simple but oddly there isn't much information out there on how to do it. A few people mentioned that changing the agent from Hosted to Hosted VS2017 seemed to do the trick for them but the results were the same for me regardless of the agent I used. The agent always stalled on the test step.

One other thing to mention is that I noticed a lot of recent posts stating that something had changed recently with the hosted agents, and test tasks that used to work were now failing or stalling. This made me think about what would happen if, for some reason, Microsoft ever decides to remove PhantomJS from the hosted agents. I figured that my builds would start failing again so, just for kicks, I changed PHANTOMJS_BIN to point to my project's local copy of phantomjs to see what would happen. It worked!

It appears that you can use either `C:\NPM\Modules\PhantomJS.cmd` or `$(Build.SourcesDirectory)\<MyProject>\node_modules\.bin\phantomjs.cmd`. I'm inclined to use the latter since it means I depend less on software that Microsoft installs, updates, or removes from the hosted agents.

So just to recap, the solution has three steps:

  • Setup your karma config to use PhantomJS as the browser.
  • Setup your gulp task to exit the process when it's done.
  • Add the PHANTOMJS_BIN build variable and point it at phantomjs.cmd.





Subscribe Here!