| Compatibility: | IdSurvey 8.0 | IdSurvey 8.9 |
| X | √ |
Supervisor is the sophisticated tool of IdSurvey for monitoring telephone interviewers’ productivity.
Two different sections are available:
Supervisor dashboard
Supervisor dashboard allows you to monitor in real-time the activity of phone interviewers made on the current date.
In addition to various productivity metrics, special “Real-Time” fields are available to display the position in the current interview and relative elapsed time.
Supervisor offers the Private Messages and Broadcast Messages tools for communicating with the interviewers, and the Watcher tool for watching the agent’s CATI window live. You can learn more on Private Messages, Broadcast Messages and Watcher in this article.
The Supervisor page is dynamic: all data updates in real time. All surveys currently in progress can be displayed in one screen.
Click on the Supervisor icon in the main menu to access the section.
CATI Activity
CATI Activity allows you to analyze the log of activities performed by interviewers in a specific survey.
You can set a date range to show only data for a specific period of time.
The data displayed in CATI Activity is static and therefore not updated in real time.
You can access CATI Activity from the Statistics menu of each individual survey.
Differences
Supervisor dashboard and CATI Activity offer specific tools that are only available in one of the two sections. Specifically:
All Supervisor data is recorded in a new dedicated log that includes details for accurate calculation of outcomes, work time, connection time and other metrics. As a result:
- Data is not available for activities performed prior to the upgrade to IdSurvey 8.1.
- The data may differ significantly from those of the old “Interviewers Activity” section. In the next versions of IdSurvey, the Interviewers Activity page will be replaced by CATI Activity, with basic functionalities available to all users even without a Supervisor license.
- Every time any interviewer activity is recorded, timing metrics in CATI Activity (Connection time, Working time, etc…) are calculated incrementally. So the timings of the currently connected interviewers are to be considered partial until the end of the working session. To visualize the accurate timing of the current day it is therefore advisable to use the Supervisor dashboard.
User Interface
Survey selection
By default, Supervisor shows all surveys with at least one activity on the current date (All surveys). By clicking on the selector you can choose a single survey among those in progress.
Group / Ungroup
When the selector is set to “All surveys” you can choose whether to show the activities grouped by single survey (Group) or to show them in a table (Ungroup).
In Ungroup mode the Survey ID column is shown and the totals row is not available.
Basic visualization and extended table
You can switch between basic view and extended table at any time. The two visualizations can be independently configured. For example, you can monitor the main data in the basic view and the more detailed data in the extended table.
Basic
The basic view allows you to monitor a maximum of 10 columns of data.
Some columns are predefined and cannot be deleted, moved, or replaced. The first two columns are dedicated to Interviewer Name, and Survey ID (the latter visible in “Ungroup” mode only).
The last column is designated for the Interviewer Score, the unique instant-read system that measures the interviewer’s performance against the survey average.
Extended table
The table offers maximum flexibility, allowing the supervisor to set an unlimited number of columns to monitor. Unlike the list view, all columns are customizable, with no exceptions.
Add a metric
To add a metric right-click on any column heading, choose “Add new” and then select the data to be added to the visualization.
The new column will be created to the right of the one you selected.
Replace a metric
To replace metric in the column, right-click on any column header, select “Choose data” and then choose the desired data from the menu.
Move a column
To move a column, click and hold the column header. Move the header to the desired position and release the mouse button.
Delete a column
To delete a column, right-click in the header, then choose “Delete”.
Order
To sort rows by values, click on the desired column header. Click a second time to change the order from ascending to descending or vice versa.
The order is one-shot: the arrangement of the rows is not changed when the data are updated. To force the update of the previously set order, click on the column header.
Today / Current sessions
The Current Sessions filter allows you to view the activities of currently connected interviewers. All data are related to the most recent current session only.
The Today filter (default) shows all activities performed on the current date. All data is related to the entire working day, regardless of the number of sessions performed by the agent.
Each row displayed in the table refers to the activities the interviewer performed on a given survey. The activities of interviewers who worked on more than one survey during that day are listed separately, each in the corresponding survey. When setting the Today filter and monitoring “All surveys”, you may find the same interviewer. However, the data in each row refers to the activities performed in that particular survey.
Options
Some settings can be configured from the gear menu.
Elapsed time highlighting threshold
Elapsed time displays the time elapsed since the last page change made by the interviewer. Once the threshold is exceeded, the data is highlighted in red to draw the supervisor’s attention. By using the tools menu you can set the threshold with a value between 1 and 10 minutes.
Interviewer Score threshold and mode
By using the colored slider you can customize the thresholds that determine the color change of the Interviewer Score.
It is also possible to change the analysis mode of the Interviewer Score between Completed per Hour (default) and Completed per Attempts. You can find all the details in the Interviewer Score paragraph.
Duration fields format
This option allows you to choose how to display the fields containing a duration. You can choose the humanized format (default), e.g. 1 hour 25 minutes and 7 seconds, or the compact format, e.g. 01:25:07.
Views
Similar to other sections of IdSurvey, Supervisor also provides the Views tool for saving settings. Any configuration of the Supervisor page can be saved in the view: displayed columns, options, sorting, analyzed survey, etc…
The last view you used is automatically loaded when logging in.
By clicking on the view menu you can retrieve one of the default, team, or private views.
Default views
They are available to all supervisors who have access to the page but cannot be overwritten or deleted.
Team views
They are available to all supervisors who have access to the page, but they can be modified and deleted only by the supervisor who created them.
Private views
They can be seen, edited, and deleted only by the supervisor who created them.
If you make any changes, the view name is replaced with “View not saved” and the Save button is displayed. All unsaved editing is still stored in the temporary “View not saved”, which is then automatically loaded the next time you log in.
If the source view was created by the same supervisor, the system asks if the view has to be overwritten or a new one created.
If the source view is the default one, created by another supervisor or a new one, you are prompted to specify a name and select the team or private type.
Alternatively, to edit or delete an existing view, you can enter edit mode from the views menu by clicking on the tools icon next to the name of the view you created.
Tools
Idle detection
In Supervisor you can not only display the connection time of the interviewer but also the working time and inactivity time. Idle detection is the system that detects the inactivity of the interviewers.
How it works
If the interviewer does not perform any activity or no mouse movement is detected for a few minutes, IdSurvey activates Idle detection.
The Supervisor dashboard shows the agent in idle and the time is considered as Idle time instead of working time.
At the same time a notification is displayed in the interviewer’s window.
As soon as the interviewer resumes working in the IdSurvey window – simply by moving the mouse – Idle detection is immediately deactivated: idle time stops and work time resumes.
If the interviewer is inactive for 30 minutes, the system performs an automatic logout.
For greater data accuracy, Supervisor translates the last Idle detection event as a logout, updating the connection time and idle time.
This ensures greater accuracy and reliability of connection, work, and idle times, even when the agents forget to log out at the end of their shift.
The Idle detection feature is not available in the waiting pages because long periods of inactivity can be very common. For example, For example, waiting for a new contact to be passed from Predictive Dialer, selecting/searching contacts in inbound mode, and when the No contacts available page is shown in outbound mode.
In any case, Supervisor provides specific metrics to monitor the time interviewers spend on waiting pages (Waiting time category).
Interviewer Score
Interviewer Score is a unique tool that shows each interviewer’s current productivity compared to the survey average. Interviewer Score compares the interviewer’s productivity in the period analyzed to the overall productivity of all interviewers for the entire period of survey activity.
The result of the calculation is expressed as a percentage. If the interviewer has an average of half as many completions per hour in the current session as the average for the entire survey, the score will be -50%. If the interviewer has an average of twice the survey average, the score will be +100%.
For easy interpretation, the Interviewer Score ranks the productivity using three colors.
Green: above average
Yellow: average
Red: below average
By default, the thresholds that determine the color change are set to -33% and + 33%.
You can customize the ranges according to your needs using the gear menu. In addition to the hourly average calculation, it is possible to set the Interviewer Score by attempts.
The score calculation will be based on the ratio of completed interviews to the number of attempts. To display the specific percentage of an interviewer simply move the mouse over the relative colored symbol.
Watcher
Watcher allows you to watch the interviewer’s working window live.
If the interviewer is connected, the button to access the screen will be available in the line.
The screen shows the name or username of the interviewer being watched, the name of the survey, and a replica of the CATI window.
Watcher’s technology does not require any additional software or plugins.
Watcher is designed to protect the privacy of interviewers. In fact, only the CATI interface is shared, ignoring any other window or activity that the interviewer may do at the workstation.
This key feature makes Watcher suitable also for companies or organizations that implement special privacy regulations for their collaborators or employees.
Private messages
Private messages allow you to chat with your interviewers. The operation is intuitive and similar to the messaging apps we all use on smartphones.
By clicking the “Private messages” button on the home screen, you can see the conversations that have already started.
By clicking on a conversation, all messages with the agent are shown. You can write and send a new message in the input box at the bottom of the screen.
Click the “+” icon from the main screen, then search and choose the desired interviewer to start a new conversation.
Connected agents are marked with a green dot next to their names.
The agent or supervisor receives a notification at the bottom of the screen that disappears after a few seconds. The conversation can also be opened by clicking on the on-screen notification.
A badge with the number of unread messages is also displayed.
The Unread status is shown below the messages that the interviewers haven’t read yet along with the time of sending. Neither the Unread status – if any – nor the supervisor’s name is shown in the interviewer’s interface.
Broadcast messages
Broadcast messages are designed for sending communications to the interviewers working on a given survey.
You can create 2 types of broadcast messages:
- Red: to all interviewers connected or who will connect to a given survey.
- Yellow: only to interviewers currently connected to a given survey.
These two types can be used to differentiate messages with instructions related to the survey (red) – such as “Interview only the owner of the company” – to those with instructions related to the work session (yellow) – such as “Stop the work of this survey at 6 p.m. and continue with survey X.”
Broadcast Messages are unidirectional: the interviewers cannot respond to these messages.
To create a new Broadcast message, you must choose a survey from the list on the main screen.
When you click on the survey, any previously published broadcast messages are shown.
By clicking on an existing message, you can access the details to see which agents have received it or to store the message. Stored messages are no longer visible to agents, but they can still be accessed by supervisors.
Click on the “Add new Broadcast message” button to add a new message.
Write the message, choose if you want to send it only to connected interviewers, and finally confirm by pressing Publish.
Metrics and fields
Supervisor provides over 80 metrics and metadata to monitor every aspect of your interviewers’ work.
By right-clicking a header you can change the current column metric or add a new one.
The hierarchical menu – complete with a search field – makes the desired metric or data easy to find.
Call outcomes
You can choose any CATI outcome you want to monitor. The outcomes are divided into categories, each of them includes a special data element that represents the sum of the outcomes belonging to it.
| Attempts | Total number of attempts |
| Completed | Number of “complete interview” outcomes |
| Unsuccessful calls (all) | Sum of Unsuccessful calls category outcomes (includes No answer, Busy line, Abandoned, Disconnected, Voicemail, Fax, Out of service, Wrong number) |
| No answer | Number of “No answer” outcomes |
| Busy line | Number of “Busy line” outcomes |
| Abandoned | Number of “Abandoned” outcomes (only available with Predictive Dialing) |
| Dropped call | Number of “Disconnected” outcomes |
| VoiceMail | Number of “Voicemail” outcomes |
| Fax | Number of “Fax” outcomes |
| Out of service | Number of “Out of service” outcomes |
| Wrong number | Number of “Wrong number” outcomes |
| Appointments and CAWI invitations (all) | Sum of Appointments and CAWI invitation category outcomes (includes Appointment, Email sent, SMS sent) |
| Appointment | Number of “Appointment” outcomes” |
| Email sent | Number of “Email Sent” outcomes” |
| Sms sent | Number of “SMS sent” outcomes” |
| Refusals (all) | Sum of Refusals category outcomes (includes Refusal from the Start, Refusal During Interview, Privacy Not Accepted) |
| Refused from the start | Number of “Refused from the start” outcomes (Respondent’s refusal given on the intro page) |
| Refused during interview | Number of “Refused during interview” outcomes (Respondent’s refusal given during the interview) |
| Privacy not accepted | Number of “Privacy not accepted” outcomes |
| Out of quota / target (all) | Sum of Out of quota / target category outcomes (includes Out of quota, Out of target) |
| Out of quota | Number of “Out of quota” outcomes |
| Out of target | Number of “Out of target” outcomes |
Timing
“Timing” data allows you to monitor several indicators of the interviewers’ working time.
Connection time reports the time spent by an interviewer in a given survey. The connection time is calculated from the moment the interviewer logs into the survey until the logout – which can be manual or automatic.
The connection time is divided into working time and idle time. IdSurvey automatically detects idle time by using the Idle detection function.
Other indicators are also available to monitor the time spent on the intro, questionnaire, confirmation, and waiting pages (Predictive, Inbound, and No contact available).
| Connection time | This is the total interviewer connection time. The connection time is equal to the sum of the working time and the idle time. |
| Working time | This is the interviewer’s total working time, excluding idle time. |
| Idle time | This is the total amount of the interviewer’s idle time. Idle time is calculated thanks to Idle detection, the system that notices when the interviewer makes no activity in the CATI interface for a few minutes. |
| Waiting time (all) | This is the sum of the waiting times on the pages before a contact is selected. Depending on the call mode, the waiting pages are: • Call waiting from Predictive Dialing These waiting times are considered as working time and are therefore included in Working Time |
| Predictive waiting time | This is the total amount of time spent on the new contact waiting screen when working in Predictive Dialing mode. Predictive waiting time is considered as working time and is therefore included in Working Time. |
| Inbound waiting time | This is the total amount of time spent in the search and selection screen of a contact in Inbound mode. Inbound waiting time is considered as working time and is therefore included on Working Time. |
| No contacts available waiting time | This is the total amount of time spent on the “No contacts available” screen when working in Outbound mode. No contacts available waiting time is considered as working time and is therefore included in Working Time. |
| Network waiting time | It is the waiting time at the server response in the loading of the CATI interface pages. The network waiting time is considered as working time and is therefore included on Working Time. |
| Time spent in the interview (all) | It is the sum of all the time spent in the interview and includes the intro, questionnaire, and confirmation pages. |
| Intro time | This is the total amount of time spent on the intro pages. |
| Questionnaire time | This is the total amount of time spent on the questionnaire pages, including any privacy acceptance and “request data” pages. |
| Confirm time | This is the total amount of time spent on the interview confirmation pages. |
Interviews duration
Interview duration data allows you to track the average, minimum, and maximum duration of interviews conducted by an interviewer.
Interview duration refers only to the time spent on the questionnaire and does not include the intro and confirmation page.
The only interviews taken into consideration are those that were initiated and completed during the period analyzed and were carried out entirely by a single interviewer. Completed interviews that were re-opened and completed again are also excluded.
| Min interview duration | This is the shortest interview duration among those completed by the interviewer. |
| Max interview duration | This is the longest interview duration among those completed by the interviewer. |
| Average interview duration | This is the average duration of interviews completed by the interviewer over the period analyzed. The totals row shows the weighted average. |
Average
Supervisor provides a variety of metrics for tracking hourly or daily productivity. You can view averages of completed interviews, attempts, or each category of outcomes.
Metrics are also available for analyzing average connection, work, or idle times per workday.
For this category, the totals rows always show the value for the entire survey. For example, Completed per hour is calculated as Total completed interviews divided by total connection time.
| Completed per hour | This is the average number of interviews completed in one hour. Completes / Hours of connection |
| Attempts per hour | This is the average number of attempts made in an hour. Attempts / Hours of connection |
| Unsuccessful calls per hour | This is the number of attempts in the category of the Unsuccessful calls that are performed on average in one hour. The Unsuccessful calls category include the outcomes No answer, Busy line, Abandoned, Disconnected, Voicemail , Fax, Out of service, Wrong number. |
| Appointments and CAWI invitations per hour | This is the number of attempts in the Appointments and CAWI invitations category that are performed on average in one hour. The Appointments and CAWI invitations category includes the outcomes: Appointment, Email sent, SMS sent. |
| Refusals per hour | This is the number of attempts in the Refusals category that are performed on average in one hour. The Refusals category includes the outcomes Refusal at the start, Refusal during, Privacy not accepted. |
| Out of quota / target per hour | This is the number of attempts in the Out of quota / target category that is performed on average in one hour. The Out of quota / target category includes Out of quota, Out of target outcomes. |
| Completed per day | This is the average number of interviews completed in a day. Completes / Days Worked |
| Attempts per day | This is the average number of attempts made in a day. Attempts / Days worked |
| Unsuccessful calls per day | This is the number of attempts in the Unsuccessful call category performed on average in a day. The Unsuccessful calls category includes the outcomes No answer, Busy line, Abandoned, Disconnected, Voicemail, Fax, Out of service, Wrong number. |
| Appointments and CAWI invitations per day | This is the number of attempts in the Appointments and CAWI invitations category made on average in a day. The Appointments and CAWI invitations category includes the outcomes Appointment, Email sent, SMS sent. |
| Refusals per day | This is the number of attempts in the Refusals category made on average in a day. The Refusals category includes the outcomes Refused from the start, Refused during the interview, Privacy not accepted. |
| Out of quota / target per day | This is the number of attempts in the Out of quota / target category made on average in a day. The Out of quota / target category includes Out of quota, Out of target outcomes. |
| Connection time per day | This is the average connection time per workday. Connection Time / Work Days |
| Working time per day | This is the average work time per workday. Work Time / Work Days |
| Idle time per day | This is the average idle time per workday. Idle Time / Days Worked |
Extra
The Extra category provides special additional metrics and Interviewer Score.
| Interviewer Score | It is a unique tool that uses different colors to shows each interviewer’s current efficiency compared to the survey average. Interviewer Score relates the interviewer’s productivity over the analyzed period to the overall productivity of all the interviewers on all days of survey activity. |
| Processed contacts | This is the number of unique contacts processed by the agent. For example, if the contact John Smith has been contacted multiple times, generating 3 attempts (busy number, then appointment, then full interview), it is counted only once in Processed Contacts. However, if attempts to contact John Smith were made by multiple interviewers, Processed contacts count the contact to each operator who processed it. |
| Days of work | This is the number of days in which the interviewer performed at least one activity during the period analyzed. The totals row shows the number of days on which the survey recorded at least one activity, regardless of the number of interviewers who worked on it. |
| Single completed interviews | This is the number of completed interviews net of re-opened and re-completed interviews. In the attempts category, there is a Completed counter that records the number of “interview complete” outcomes done by the interviewer. In some cases, such as for revisions and corrections, it may be necessary to re-open an interview and then mark it as completed again. By their nature, the two “complete interview” outcomes are independent. This means that if a completed interview is re-opened by an administrator and then completed again, the Completed entry counts 2 separate outcomes, even though they refer to the same interview. Single completed interviews report the exact number of completed interviews excluding from the count any new Completed outcomes that occurred as a result of a reopening. |
Real-Time
Call method, timing, and location of the interview currently being processed by the interviewer. This data is only available if the interviewer is connected to the survey.
Interview progress does not take into account display conditions, skips, section loops, or previous questions editing and may therefore differ significantly from the actual advancement of the interview.
| Position | This shows the current page that the interviewer is browsing. If the interview is in progress, it shows the code of the questionnaire page. It also shows the Idle label when the interviewer has been inactive for about 2 minutes. Possible positions can be: Inbound waiting, Outbound waiting, No contacts available waiting, Intro, Privacy, Contact Details, Questionnaire page code, Confirm. |
| Elapsed time | This shows the elapsed time since the last position change. If the elapsed time exceeds 2 minutes, it is highlighted in red to draw the supervisor’s attention. By using the gear menu you can customize the highlighting threshold from 1 to 10 minutes. |
| Call mode | This displays the interviewer’s current call mode: Outbound, Predictive Dialing, or Inbound. |
| Time on current contact | This shows the elapsed time in the current contact. In other words, the time passed since the intro page of the currently processed contact was shown. |
| Interview progress | It indicates the percentage of progress in the current interview. |
Metadata
These fields show key information about the survey and the interviewer.
| ID Survey | This is the survey ID number. |
| Survey title | This is the survey title. |
| Folder | This is the name of the folder containing the survey. |
| ID | This is the interviewer ID number. |
| Interviewer | This shows the name and surname of the interviewer, if not available, only the username is shown. |
| Interviewer’s fields Name Surname Username Identification Type Phone number Mobile Address Postcode City Gender Tax code Birth date Date of hiring Agent Extension | They display the values available in the corresponding interviewer fields. |
| Groups | This is the list of groups the interviewer belongs to. |