Merging Helpdesks, A Performance Analysis
June 23, 2014 § 2 Comments
I recently encountered a scenario where two helpdesks were to merge. One large, with relatively poor response times, and a smaller one, with relatively good performance. The question was, what would be the impact of merging the call queues on the current customers of the smaller helpdesk? It might seem that the performance of the smaller helpdesk would necessarily be significantly harmed. I’m going to show that that isn’t necessarily the case.
Neil Gunther’s PDQ::Perl1 can be used to do a Mean Value Analysis and predict the impact of combining the queues. That is to say, work out the average performance. I wrote a small Perl script to do this, documented here:
https://ascknd.com/2014/06/23/helpdesk-queue-analyser-pdqperl/
Paradoxically, what we find is that, under certain circumstances, merging the two queues delivers better performance than either helpdesk was capable of delivering on its own:
Example:
Site A has 4 staff working on their queue (e.g. NetApp support), with a call coming in on average every 12.5 minutes. It takes them 30 minutes to deal with each case (excluding time spent in the queue waiting for an operative to become free). At site B they have 6 staff dedicated to a similar queue, who also take 30 minutes to deal with each case, but have an incoming call rate of one call every 5.6 minutes.
Using the previously mentioned script, the behaviour of these helpdesks can be analysed with the following parameters:
perl ./helpdesk_response.pl 4 30 0.08 6 30 .18
What we find is that Site A is 60% utilized and typically only keeps calls waiting in the queue for just over 5 minutes. Site B is 90% utilized and keeps callers waiting in the queue for 37 minutes. When the two queues are combined both sites are 78% utilized and experience waiting times in the queue of <5 minutes. So both sites experience better performance.
How did merging the work of a site with good performance and a site with relatively poor performance lead both sites to improve? Two things are going on:
1. The impact of utilization on performance is non-linear. Moving from 60% utilization to 78% utilization does relatively little harm to site A compared to the benefit experienced by site B.
2. More helpdesk operators are more efficient, from a queueing perspective, than fewer helpdesk operators given the same utilization. That is to say, 6 workers at 60% busy will be more responsive than 4 workers at 60% busy even though in both cases the individual helpdesk operators are performing identically. This is because the more operators there are, the greater the chance that one of them will be free when a call comes in.
Conclusion:
In the general case, one can’t assume that combining the queues of the two helpdesks will necessarily cause the performance of a smaller high performing helpdesk significant harm.
Helpdesk Queue Analyser PDQ::Perl
June 23, 2014 § 1 Comment
Below is a Perl script using the PDQ Perl module to analyser the performance of two helpdesk and predict their behaviour should their queues be merged.
A more complete discussion of the motivations for this and conclusions can be found here:
#/usr/bin/perl
use pdq;
use strict;
use warnings;
my $siteAStaff = $ARGV[0];
my $siteAServiceTime = $ARGV[1];
my $siteARate = $ARGV[2];
my $siteBStaff = $ARGV[3];
my $siteBServiceTime = $ARGV[4];
my $siteBRate = $ARGV[5];
#Handle site A
pdq::Init(“Open Network with M/M/N”);
pdq::SetComment(“Simulation of site A mean performance.”);
pdq::CreateOpen(“work”, $siteARate);
pdq::CreateMultiNode( $siteAStaff, “cores”, $pdq::CEN, $pdq::FCFS);
pdq::SetDemand(“cores”, “work”, $siteAServiceTime);
pdq::SetWUnit(“Calls”);
pdq::SetTUnit(“Mins”);
pdq::Solve($pdq::CANON);
my $responseA = substr(pdq::GetResponse($pdq::TRANS, “work”),0, 8);
my $utilA = substr(pdq::GetUtilization(“cores”, “work”, $pdq::TRANS) * 100,0, 4);
#Handle site B
pdq::Init(“Simulation of site B mean performance.”);
pdq::SetComment(“Simulation of 8 core CPU under transactional load.”);
pdq::CreateOpen(“work”, $siteBRate);
pdq::CreateMultiNode( $siteBStaff, “cores”, $pdq::CEN, $pdq::FCFS);
pdq::SetDemand(“cores”, “work”, $siteBServiceTime);
pdq::SetWUnit(“Calls”);
pdq::SetTUnit(“Mins”);
pdq::Solve($pdq::CANON);
my $responseB = substr(pdq::GetResponse($pdq::TRANS, “work”),0, 8);
my $utilB = substr(pdq::GetUtilization(“cores”, “work”, $pdq::TRANS) * 100,0, 4);
#Handle combined site
pdq::Init(“Open Network with M/M/N”);
pdq::SetComment(“Simulation of comined site mean performance.”);
pdq::CreateOpen(“work”, $siteARate + $siteBRate);
pdq::CreateMultiNode( $siteAStaff + $siteBStaff, “cores”, $pdq::CEN, $pdq::FCFS);
pdq::SetDemand(“cores”, “work”, ($siteAStaff * $siteAServiceTime + $siteBStaff * $siteBServiceTime) / ($siteAStaff + $siteBStaff) );
pdq::SetWUnit(“Calls”);
pdq::SetTUnit(“Mins”);
pdq::Solve($pdq::CANON);
my $responseAB = substr(pdq::GetResponse($pdq::TRANS, “work”),0, 8);
my $utilAB = substr(pdq::GetUtilization(“cores”, “work”, $pdq::TRANS) * 100,0, 4);
printf “Site \t| Response \t | utilization\n”;
printf “A \t| %f \t | % 4d\n”, $responseA, $utilA;
printf “B \t| %f \t | % 4d\n”, $responseB, $utilB;
printf “Combined\t| %f \t | % 4d\n”, $responseAB, $utilAB;
Syntax:
perl ./helpdesk_response.pl <siteA call helpdesk staff#> <siteA service time (mins)> <siteA calls per minute> <siteB call helpdesk staff#> <siteB service time (mins)> <siteB calls per minute>
NB: Service time is used here, not response time, so time spent in the queue doesn’t count. Response time is used in the outputted performance.
Example:
perl ./helpdesk_response.pl 4 30 0.08 6 30 .18
Site | Response | utilization
A | 35.382050 | 60
B | 67.006280 | 90
Combined | 34.990830 | 78
Perl PDQ: 8 core response time under load
May 15, 2014 § 1 Comment
The following Perl PDQ1; script was used to generate the response time data for the 8 core example in my https://ascknd.com/2014/05/02/nfs-vs-fibre-cpu-vmware/ post:
#/usr/bin/perl
use pdq;
$cores = $ARGV[0];
$servTime = $ARGV[1];
$max_rate = (1 / $servTime) * $cores;
print “response(secs)\t | util_%\n”;
for ($rate = 0; $rate < $max_rate ; $rate += $max_rate / 5) {
$arrivRate = $rate;
pdq::Init(“Open Network with M/M/N”);
pdq::SetComment(“Simulation of N CPU cores under transactional load.”);
pdq::CreateOpen(“work”, $arrivRate);
pdq::CreateMultiNode( $cores, “cores”, $pdq::CEN, $pdq::FCFS);
pdq::SetDemand(“cores”, “work”, $servTime);
pdq::SetWUnit(“IOS”);
pdq::SetTUnit(“Secs”);
pdq::Solve($pdq::CANON);
$response = substr(pdq::GetResponse($pdq::TRANS, “work”),0, 8);
$util = substr(pdq::GetUtilization(“cores”, “work”, $pdq::TRANS) * 100,0, 4);
printf “$response \t | % 4d\n”, $util;
}
perl ./mm8_response.pl 8 0.000127
response(secs) | util_%
0.000127 | 0
0.000127 | 20
0.000127 | 40
0.000132 | 60
0.000163 | 80
ascknd 