Electronic Voting & Counting Technologies - IFES

ELECTION TECHNOLOGYSERIESElectronic Voting & Counting TechnologiesA Guide to Conducting Feasibility StudiesBen GoldsmithSenior Electoral Advisor

Electronic Voting &Counting TechnologiesBen GoldsmithSenior Electoral AdvisorInternational Foundation for Electoral Systems

Electronic Voting &Counting TechnologiesA Guide to Conducting Feasibility StudiesBen GoldsmithSenior Electoral AdvisorMay 2011Any opinions, findings, conclusions or recommendations expressed in this publication are those of the author(s) and donot necessarily reflect the views of the International Foundation for Electoral Systems.

ABOUTThe AuthorBen Goldsmith is a senior electoral advisor with over 10 years of experience advising and managing electionadministration projects in post-conflict and developing democracies. Goldsmith currently serves as IFES’chief of party in Pakistan. He manages the implementation of an electoral assistance program directed atthe Election Commission of Pakistan and Pakistani civil society. In this role he focuses on issues such asimproving the voter registry, election law unification and reform, training and capacity building, and the useof new voting technologies.Goldsmith has helped to conduct elections in Bosnia, Kosovo, Afghanistan, Iraq and the United Kingdom,including serving as the deputy chief election officer for the 2005 National Assembly and Provincial CouncilElections. He has also provided technical assistance in Georgia, Egypt, Lebanon, Nepal and Pakistan.Prior to IFES, Goldsmith held electoral roles with and on behalf of the UK Foreign and Commonwealth Office,the Organization for Security and Cooperation in Europe, the International Organization for Migration, theUnited Nations and international non-governmental organizations such as The Asia Foundation. He holdsextensive experience in a number of different regions, including Europe, South Asia and the Middle East, andhas managed projects in over 20 countries.Goldsmith holds two master’s degrees from the University of Bristol: a M.Sc. in International Relationsand a L.L.M. in International Law. Goldsmith has published numerous articles on elections, out of countryvoting and Central Asian politics.IFESThe International Foundation for Electoral Systems (IFES) is the leading election assistance and democracypromotion non-governmental organization.IFES promotes democratic stability by providing technical assistance and applying field-based researchto the electoral cycle in countries around the world to enhance citizen participation and strengthen civilsocieties, governance and transparency.Every IFES project is staffed by national and international personnel while partnering with local electionmanagement bodies and civil society organizations. This homegrown approach ensures that the expertiseoffered by IFES fits the needs of the country or client and the benefit of assistance outlasts the life of theproject. Our work is nonpartisan and includes projects that:• Help citizens participate in their democracies• Increase politicians’ accountability to the electorate• Strengthen government institutionsSince its founding in 1987, IFES has worked in 133 countries — from developing democracies to maturedemocracies.IFES is registered in the United States as a 501(c)3 organization.iiElectronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

Table of ContentsAcknowledgementsiAboutiiIntroduction 1Terminology 3Benefits & Challenges of Electronic Voting and Counting Technologies 4Electronic Voting and Counting Technology Feasibility Study 12International Electoral Standards 17Decision in Principle 25The Mandate of the Study 25The Feasibility Study Committee 27Vendor Relations 30Feasibility Study Committee Working Groups 31Issue 1 — Assessment of the Current System ofVoting and Counting 31Issue 2 — Assessment of the Advantages and DisadvantagesOffered by Voting and Counting Technologies 32Issue 3 — Review of IT Security Aspects 33Issue 4 — Determining Technical Feasibility 35Issue 5 — Cost Benefit Analysis 37Issue 6 — Institutional Capacity 41Issue 7 — Legal Reform Issues 42Study Trips 44Vendor Demonstration 45Stakeholder Consultation 47Decision in Principle 48Pilot Prerequisites 51Pilot Project Mandate 51Type of Pilot 51Pilot Locations 52Solutions Being Piloted 53Legislation 53Electronic Voting and Counting Technology Specification 54Pilot Project Funding 58International Foundation for Electoral Systemsiii

TABLE OF CONTENTSPilot Project 59Managing the Pilot Project 59Risk Management Plan 60Procuring Electronic Voting and Counting Technologies 61Testing and Certification 62Polling and Counting Procedures 63Voter Education 64Training 65Stakeholder Outreach 66Publicity 66Election Day Support 66Observation of the Pilot Project 67Mandatory Audit 67Pilot Project Evaluation 69The Decision on Adoption 75Additional Resources 79Annexes 81Annex 1 — Draft Timeline for Decision in Principle 82Annex 2 — Feasibility Study CommitteeDecision in Principle Report — Table of Contents 84Annex 3 — Suggested Briefing Pack for Feasibility Study Committee 85Annex 4 — Example of Weighted Ranking of Vendor Proposals 86Annex 5 — Sample Pilot Project Report Table of Contents 87ivElectronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

IntroductionAutomating ElectionsThe Philippines introduced electronic voting and countingtechnologies during its May 2010 national elections.In many areas of modern life today, technology dominates. It is believed that technology isprogress; progress is good and should, therefore, be embraced. An initial look at the field ofelections may lead to a similar conclusion.Electronic voting and counting technologies are being increasingly used around the world.India, the world’s largest democracy, now uses electronic voting machines exclusively fornational and provincial elections. Brazil, Belgium and the Philippines also use electronicvoting or counting technologies for all of their national elections. Countries such as Estonia,Indonesia, Kazakhstan, Nepal, Norway, Pakistan, Russia and the United States are at variousstages of piloting or partially using electronic voting and counting technologies, including theuse of Internet voting.Some countries, however, are moving in the opposite direction. The Netherlands, in 2008,after several decades of increasing use of electronic voting machines, decertified all of itsmachines and moved back to paper balloting. Germany, likewise, recently banned the use ofelectronic voting machines it had been using, and in Ireland €52 million 1 worth of electronicvoting machines were bought but only used for a small pilot project. Furthermore, the use ofelectronic voting and counting technologies in the United States is deeply controversial andgenerates fierce debate between advocates and opponents of these technologies.1 See parliamentary debate on Irish e-voting at http://historical-debates.oireachtas.ie/D/0625/D.0625.200610170016.html (last accessed on 24 January 2011).International Foundation for Electoral Systems 1

INTRODUCTIONINTERNATIONAL ELECTORAL STANDARDSHow are we to reconcile these very differentapproaches to the suitability of electronicvoting and counting technologies? For acountry considering electronic voting orcounting technologies, which is the rightapproach and when is it advisable to proceedusing these technologies?The answer is, of course, that there is noone answer. The factors which may pushone nation towards an electronic voting orcounting technology may not be present foranother nation, or may indicate a differentsolution. The challenges of moving paperballots around large countries such as Russiaand Kazakhstan make the use of electronicvoting technologies potentially beneficial onlogistical grounds. The existence of a smartID card with digital signature for the majorityof the population in Estonia makes the use ofInternet voting more feasible in Estonia. ThePhilippines adopted an electronic countingsolution to deal with issues related to fraudduring the counting process.In short, the factors that argue for oragainst the use of electronic voting orcounting technologies in a particularcountry are specific to that country andwill have many different sources — legal,cultural, political, logistical, environmental,etc. This guide attempts to provide aframework for conducting a full assessmentof whether electronic voting and countingtechnologies are appropriate for a particularelectoral process.A key theme in this guide is that transparencyand openness are essential to successfulconsideration and implementation of“The factors that arguefor or against the use ofelectronic voting or countingtechnologies in a particularcountry are specific to thatcountry and will have manydifferent sources.”electronic voting and counting technologies.Change can be unsettling, and it is crucialthat stakeholders trust the electoral process.When decisions are taken behind closeddoors without involving stakeholders, therationale for those decisions will not be clearand stakeholders will naturally question whothe change is intended to benefit. This canlead to erosion of trust in the process andseverely undermine the credibility of theelections and the elected institutions.As Thad Hall argued in a presentation atthe EVOTE2010 Conference, it is not thetechnology that is used that matters, but theway in which the technology is implementedthat ultimately determines the success ofthe election technology project. 2 This guideprovides a solid basis for the decisionmakingprocess involved in whether or notto implement these technologies. The guidestresses that proper consideration should begiven to all factors influencing the decisionwhether to adopt voting and countingtechnologies and that all stakeholders havethe opportunity to understand and expresstheir opinions during the process.2 Notes from Session 1, 22 July 2010, Thad HallPresentation on “Voting Technology and ElectionExperience: The 2009 Gubernatorial Races in NewJersey and Virginia”, EVOTE2010 Conference,Bregenz Austria.2Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTRODUCTIONINTERNATIONAL ELECTORAL STANDARDS• Increase in Turnout — Electronic votingand counting technologies may increaseturnout if these technologies helpimprove trust in the electoral process;if the technology makes people moreinterested in participating or increasesaccess for certain communities. 5“Electronic voting andcounting technologiesmay increase turnout ifthese technologies helpimprove trust in the electoralprocess.”• Elimination of Invalid/Incorrectly CastBallots — In some countries significantnumbers of ballots are deemed invalidand not counted. Those voters aredisenfranchised. Where ballots are castand recorded electronically, the electronicvoting software can be configuredto ensure only valid ballots are cast(although blank ballots may still beallowed). Likewise where paper ballotsare inserted into an electronic ballot box,the validity and choices of the voter canbe displayed, allowing voters to changetheir ballot if a mistake was made.• Speed of Counting — An importantadvantage of using electronic votingtechnology, which directly recordvotes electronically, is that results areimmediately available after polls close,without a lengthy counting process.5 For example, geographically remote communities,people with disabilities and overseas voters.Even when paper ballots are used,but electronically counted, the resultsare normally available a lot faster thanmanual counting.• Standard Adjudication of Ballots —Counting paper ballots electronicallyensures that the same kind of ballotmarking is adjudicated in the samemanner across all polling stations. Thisensures consistency on which ballots arecounted and which are determined tobe invalid. This is often not the case withmanual counting of ballots.• Accurate Tabulation of Results — Whenresults are electronically recorded andtransmitted to the election managementbody (EMB) for tabulation, the possibilityof data entry errors during resultstabulation is greatly diminished.• Impartiality — Electronic voting andcounting technologies follow predefinedrules and are independent from humaninfluence and impartial.• Fraud Prevention — Electronic votingand counting technologies can mitigatesome fraud in polling stations. Forexample, some electronic voting andcounting technologies only allow votesto be cast at a certain speed, thusmitigating ‘ballot stuffing.’ Similarly,electronic counting of ballots mitigatesfraud during the counting process.Electronic voting and countingtechnologies cannot, however, eliminateall aspects of electoral fraud.6Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL INTRODUCTIONSTANDARDS• Cost — Electronic voting and countingtechnologies remove the need forexpensive ballot printing, distribution,storage, etc. However, thesetechnologies also incur different costswhich need to assessed over the lifecycle of the technology.Disadvantages of voting and countingtechnology listed by opponents include:• Lack of Transparency — Transparencyis a key component of building andmaintaining trust in the electoralprocess. The paper balloting system isvery transparent. Observers can watchballots being issued, voters placing theirmarked ballots in the ballot box andballots being counted. Electronic votingtechnology, more so than electroniccounting technology, is often consideredto be a ‘black box.’ This is because itis not possible to observe the way inwhich the selected choices of votersare aggregated to produce the resultsannounced. We simply have to trustthat these results accurately reflect thechoices made by voters. This makesthe checking of results producedby electronic voting and countingtechnologies all the more important.• Confidence — Lack of transparencywith electronic voting and countingtechnologies means that confidencein the operation of the technologyis a considerable problem. Electionmanagement bodies need to ensurethat trust in the electoral process ismaintained. Once trust is lost, it is difficult“Lack of transparency withelectronic voting and countingtechnologies means thatconfidence in the operationof the technology is aconsiderable problem.”to re-establish. While the introductionof electronic voting and countingtechnologies does not have to lead to anerosion of trust in the electoral process,it has happened in some countries.Election management bodies are likelyto have to introduce new procedures,possibly random audit of results orpublication of source code for electronicvoting and counting technologies, inorder to maintain trust in the process.• Audit of Results — A great strength ofthe paper balloting system is that if theresults of an election are challengedthen the ballots can be recounted tocheck the result. Many electronic votingmachines 6 have no such possibility forauditing and checking the results of anelection. The ability to audit and checkis an important feature of building trustin the electoral process and increasingacceptance of the results. Someelectronic voting machines do havewhat is called a Voter Verified PaperAudit Trail (VVPAT), which prints a copyof the electronic ballot and is verifiedby the voter before casting the ballot.This VVPAT can be used to audit/check electronic results produced bythe electronic voting machine (EVM).6 Electronic counting machines have the paper ballotcompleted by the voter as a natural audit trail.International Foundation for Electoral Systems 7

INTERNATIONAL ELECTORAL INTRODUCTIONSTANDARDSConsiderations such as storage and maintenance of voting andcounting technology may be just as important as cost.concern, it is worth noting that simplerelectronic voting and counting solutionshave been successfully used forpopulations with high levels of illiteracy.• Digital Divide — Access that somevoters may have to new votingtechnology, especially Internet votingtechnology, may serve to exclude somesections of the community which donot have such similar access to casttheir ballot. This may increase barriersto participation amongst poor, illiteratevoters and violate the principle of equalaccess to the electoral process for alleligible to participate.• Voter Education — A considerableamount of voter education would berequired to educate and prepare votersfor a move to electronic voting technology,and to a lesser extent electronic countingtechnology. This voter education exercisewould likely be costly.• Specialized IT Skills — Maintenance andrepair of hardware used by electronicvoting and counting technologiesrequires specialized IT skills which mayor may not be available in sufficientsupply and at a reasonable cost in thelocal labor market. These skills may berequired centrally as well as at the locallevel in order to deal with problems closerto Election Day if field based electronicvoting or counting machines are used.More specialized IT skills may even berequired at the polling station in order tooperate any electronic voting or countingtechnology being implemented there. Ifthese skills are in short supply then theuse of electronic voting and countingInternational Foundation for Electoral Systems 9

INTRODUCTIONINTERNATIONAL ELECTORAL STANDARDStechnologies may either be unsustainableor may require the expensive import offoreign expertise.• Integrity and Accuracy of SourceCode — Electronic voting and countingtechnologies rely on software to function.This software is a set of instructions tothe electronic voting or counting systemdefining how it operates. As with anyset of instructions, mistakes can bemade and a thorough review of thesource code has to be conducted beforeusing any electronic voting or countingtechnologies. As it takes specializedtechnical skills to be able to read andunderstand source code, an independenttesting authority may be required toreview any electronic voting or countingsystem. This review would determine,to the greatest extent possible, whetherthe system is functioning according to itsspecifications and whether the systemperforms sufficiently well before it isaccredited for use in an election.• Storage of Equipment — Someelectronic voting and counting systemhardware is required to be stored undertemperature controlled conditionsbetween elections. Temperaturecontrolled storage may be difficult andcostly to find, especially on a regional orlocal basis.• Environmental Considerations —Electronic voting and countinghardware, especially the machinery,may be required to withstand andperform reliably under a wide range ofenvironmental factors including extremeheat, cold, humidity and dust. Findingelectronic voting and counting solutionswhich reliably operate in such situationsmay be difficult.• Power Considerations — Electronicvoting and counting technologies requirea source of power, with most running onmains electricity. For solutions based inpolling stations, chronic power shortagesor the lack of electricity entirely couldrequire electronic voting or countingmachines to run for the entire period ofpolling on an alternative power source.Such power requirements limit theoptions available.• Security — Different security challengesare presented by electronic voting andcounting technologies compared topaper balloting systems. For example,electronic transmission of results fortabulation presents the possibility for thesystem to be hacked and false results beinserted. Secure systems of protectionand verification for electronic data needto be ensured.• Consequences of Fraud — Whilefraud conducted using the paperballoting system is often localized andnot widespread, the possibility existswith electronic voting and countingtechnologies for fraud to be implementedon a nationwide scale. Electronicvoting and counting software could bemanipulated to record vote preferenceswhich are different from those made by thevoters, or fraud and manipulation could10Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL INTRODUCTIONSTANDARDSoccur in the electronic tabulation of resultsif such tabulation occurs directly from theelectronic voting or counting machines.“The electoral environmentalso determines, to someextent, the advantages anddisadvantages that may beexperienced.”• Management Complexity — Managingthe introduction, testing, deployment,retrieval and security for electronictechnologies can be more complicatedthan managing a paper-based election.Election management bodies often lackadequate experience in management ofsuch complex systems. This can leadto a heavy reliance on the technologycontractor to the point of surrenderingcontrol of the electoral process to aforeign entity.• Cost — The cost of electronic voting andcounting machines ranges from $300per unit for the more simple solutions toapproximately $5,000 per unit for morecomplex solutions. When aggregatedfor an entire election this can representa potentially huge investment for manycountries, although a full comparisonagainst the costs of paper ballotingneeds to take into consideration the lifecycle of electronic voting and countingtechnologies and the number of electioncycles they would be expected to cover.The relevance of these advantages anddisadvantages vary significantly betweenelectoral situations. Different electronic votingand counting technologies will be morelikely to realize different possible advantagesand be faced with different challenges.No solution is likely to realize all possibleadvantages listed above or to suffer from allof the disadvantages.The electoral environment also determines,to some extent, the advantages anddisadvantages that may be experienced.Logistical challenges may not be significantin a geographically small democracy suchas Luxembourg, but could be of criticalimportance for a large country like Russia,which has many isolated communities.Countries which have no, or very poor,means of voter identification would likelynot be able to consider remote voting,such as Internet voting. However, thosewith smartcards with personal identificationnumbers and wide Internet access, such asEstonia, might consider the challenges ofInternet voting to be manageable — at leastas an alternative voting channel.It is also worth noting that as electronicvoting and counting technologies changerapidly, the list of possible advantagesand disadvantages will also change. Thislist provides examples of various issuesa feasibility study may consider. Thechallenge for a good feasibility study will beto balance advantages and disadvantagesin the particular electoral context anddetermine whether it is possible and/orbeneficial to introduce electronic voting andcounting technologies.International Foundation for Electoral Systems 11

INTRODUCTIONINTERNATIONAL ELECTORAL STANDARDSElectronic Voting andCounting TechnologyFeasibility StudyThe decision to introduce an electronic votingor counting system for an election is complexand should not be rushed. The full processof adopting such technologies, if that is thedecision, is likely to take years rather thanmonths. The first stage in the process ofconsidering the adoption of electronic votingand counting technologies is conducting afeasibility study, defined below:“A general term that refers to various typesof systematic evaluations carried out tobetter assess the desirability or practicality offurther developing a proposed action. Suchstudies are typically performed during theplanning stages.” 9Each feasibility study is different. Thecomponents of a feasibility study and theorder in which they are completed mayvary from one context to another. Thecomponents and the order presented in thisguide are recommendations and should beadapted for the particular electoral context.Nevertheless, these components arepresented here as a model of good practicefor the conduct of feasibility studies into thepossibility of using electronic voting andcounting technologies. It is worth notingthere are many steps to conducting a goodfeasibility study as identified in this guide.It is important to understand that reachingan informed decision on the possible use ofvoting and counting technologies may takeconsiderable time.In fact, adopting electronic voting andcounting technologies is a complex decisionand will need the appropriate time andresources to be made available. Any attemptto take short cuts in this deliberation processmay result in adopting a technology whichdoes not suit the electoral context in questionor in taking a decision without the supportof key stakeholders. Either of these resultsmay seriously undermine the credibility andlegitimacy of the electoral process.Any experimentation with new technologiesshould start on a small scale andbe provided ample time for properconsideration, as clearly stated in anIFES publication on election technology. 10Entering the feasibility study processwith a target in mind for when full scaleimplementation will take place may benatural from a planning perspective, butis a dangerous approach to the feasibilityprocess. In effect it predetermines that (1)technology will be found to be feasible and(2) ignores the fact that the time it takes toconduct a good feasibility process cannotbe determined at the beginning.This guide suggests four key stages in thedecision making process for the adoption ofthese technologies.9 See this definition at http://www.i395-rt9-study.com/glossary.html (last accessed on 30 January 2011).10 Yard, M. (ed.) (2010) Direct Democracy: Progress andPitfalls of Election Technology, p. 18-19.12Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL INTRODUCTIONSTANDARDSStage 1: Decision in Principle — Thereare four components to taking a decision inprinciple about the possibility of introducingelectronic voting and counting technologies.• Technical Feasibility — Given the objectivesidentified for introducing electronicvoting and counting technologies andany existing infrastructure limitations, adecision can be made as to whetherintroduction is possible and/or meets therequirements of the electoral environmentfrom a technical perspective.• Beneficiality — Each electronic votingand counting solution will have particularadvantages and disadvantages. Eachsolution will meet the requirements forchange in different ways. The balanceof advantages to be offered, theimportance of those advantages andthe disadvantages and risks involvedin using electronic voting and countingtechnologies will lead to an overallassessment of how beneficial theintroduction of this technology could be.• Financial Feasibility — An assessmentof comparative costs of the currentsystem of balloting compared to theestimated costs of electronic votingand counting technologies needs to beconducted. Even when an electronicvoting or counting system is found tobe more expensive, this may not meanthat the introduction of such technologyis not financially feasible. On the otherhand, the assessment may show thatthe additional costs of introducingtechnology are so in excess of existingcosts as to be beyond the resourcesavailable to the EMB.• Stakeholder Acceptance — Evenwhen an electronic voting or countingtechnology was found to be beneficial,technically feasible and financiallyfeasible, it would be a brave, if notfoolhardy, EMB that proceeded with theimplementation of this technologies inthe absence of stakeholder support. Theperception of credibility is as importantfor electoral processes as actual integrity.If key stakeholders do not trust a newtechnology then they are unlikely toaccept the election results generated,creating a deficit of legitimacy for theelected institutions. Therefore, animportant component of the decision inprinciple will consist of an assessment ofthe willingness of stakeholders to acceptand trust the introduction of electronicvoting and counting technologies.The combination of these four componentsleads to an overall decision in principle.The technical feasibility assessment mayindicate there are no products availablewhich meet the requirements of the electoralprocess. The resulting decision should bethat the use of electronic voting and countingtechnologies is not feasible with the currentproducts available. This does not mean thatthe consideration of electronic voting andcounting technologies should cease entirely.New products are being developed on aregular basis and can be measured againstthe requirements identified through thefeasibility study to see if the technical feasibilitycriteria are met at some point in the future.International Foundation for Electoral Systems 13

INTRODUCTIONINTERNATIONAL ELECTORAL STANDARDSWhen an electronic voting and countingsolution is technically feasible, there still maybe grounds for deciding not to pursue it if theadvantages are minimal, the disadvantagesare significant, the risks are higher, the costsare far greater than the advantages or there issignificant stakeholder resistance to the newtechnology. The way in which these factorsare balanced will be entirely dependent on theparticular electoral environment. Countries orelection management bodies with significantresources may be willing to spend a lot moreto obtain an electronic voting and countingsolution for the same net benefits thancountries and election management bodieswith fewer resources.At the end of the day the decision will not bea purely administrative one, it may be political.The feasibility study will need to fully explorethese different components of the decision inprinciple so that all stakeholders are aware ofthe technical and financial aspects of possibleelectronic voting and counting technologiesadoption, the net benefits, and likelystakeholder reaction to the technologies.While a well executed decision in principlewill go a long way in ensuring that anyintroduction of electronic voting and countingtechnologies is done in a sound manner,there is only so much that can be understoodabout electronic voting and countingtechnologies without actually using them.Therefore, before any final decision can bemade about the possible use of electronicvoting and counting technologies it isessential that the technology be piloted.“Before any final decisioncan be made about thepossible use of electronicvoting and countingtechnologies it is essential thatthe technology be piloted.”Stage 2: Pilot Prerequisites — While theinclination may be to jump straight into a pilotproject, there are a number of prerequisitesand parameters that need to be establishedbefore conducting a pilot. Such prerequisitesand parameters include the establishmentof the pilot project mandate, passage oflegislation enabling the pilot, development ofrequirements and technical specifications forthe solution to be piloted and funding for theconduct of the pilot project. Once these arein place, the next stage of the process, thepilot project can start.Stage 3: Pilot Project — The purpose ofa pilot project is to demonstrate whetheran idea or concept is feasible in practice.In the context of an electronic voting andcounting feasibility study it should be used todetermine the following:• The solution(s) being piloted operates asexpected• The benefits anticipated can be achieved• The disadvantages entailed in using thetechnology are as anticipated and can bemitigated in an acceptable way• To assess and revise, if necessary, the listof requirements for any solution to meetthe needs of the electoral environment14Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTRODUCTIONmaking details of the process accessible, willensure that sufficient information is availablein the public arena for informed debateoutside of the mechanisms provided for bythe Feasibility Study Committee. This will helpto mitigate baseless accusations by thoseconcerned about the technologies beingconsidered.study process one by one, explainingthe kinds of issues that will need to beconsidered in implementing them.Finally, the use of electronic voting andcounting technologies has great potentialfor improving voter access to the electoralprocess. The CoE identifies a number ofgroups which may benefit from the greateraccessibility provided by such technologies,including people with visual disabilities, thosewho struggle to travel to polling stations,access for those using official minoritylanguages, military personnel overseasand citizens living and working abroad. 12However, using electronic voting andcounting technologies also has the potentialfor excluding voters, especially those whomay not understand how to use newsystems and may feel intimidated by trying todo so. The Council of Europe states that:“E-voting should result in inclusion, neverexclusion, of certain groups.” 13This is an important point to keep in mindthroughout the consideration of electronicvoting and counting technologies.After a brief discussion on internationalelectoral standards related to electronicvoting and counting technologies, the guidewill take each of these stages of the feasibility12 Ibid, p. 15-16.13 Ibid, p. 16.16Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

InternationalElectoral StandardsVoting and counting technology is valuable only to the extent that it promotesand upholds the integrity of the electoral process.When considering a change in any sort of system, especially an important one such as avoting and counting system, it is vital that the underlying standards by which differentsystems can be judged are kept in mind. There are a number of different approaches to thechallenge of judging electoral processes. In recent years, opinion appears to have coalescedaround the concept of international electoral standards as defined by public international law. 14Public international law based electoral standards are well elaborated in documents issuedby the United Nations, 15 the European Commission, 16 the Organization for Security andCooperation in Europe (OSCE) 17 and the Venice Commission. 18 The way these electoralstandards are categorized by the different institutions are not exactly the same, but it doesillustrate a common understanding of the content of international electoral standards. Drawingdirectly from the wording of Article 25 of the International Covenant on Civil and Political Rights(ICCPR), the core of these international electoral standards can be defined as the following:14 See for example the Inter-Parliamentary Union’s publication in 1994 - Goodwin-Gill, G. (1994) Free and FairElections: International Law and Practice, Inter-Parliamentary Union: Geneva and the updated version - Goodwin-Gill, G. (2006) Free and Fair Elections: New Expanded Edition, Inter-Parliamentary Union: Geneva.15 Centre for Human Rights (1994) Professional Training Series No.2: Human Rights and Elections – A Handbook onthe Legal Technical and Human Rights Aspects of Elections, United Nations: New York and Geneva.16 European Commission (2007) Compendium of International Electoral Standards: Second Edition, EuropeanCommission: Brussels.17 OSCE (2007) Election Observation Handbook: Fifth Edition, OSCE Office for Democratic Institutions and HumanRights: Warsaw.18 European Commission for Democracy Through Law (Venice Commission) (2002) Code of Good Practice inElectoral Matters: Guidelines and Explanatory Report, Adopted by the Venice Commission at its 52 nd session(Venice, 18-19 October 2002), CDL-AD(2002) 23 rev.International Foundation for Electoral Systems 17

INTERNATIONAL ELECTORAL STANDARDS• Fair Elections (without anydistinctions) — Elections should beconducted so as to ensure equalconditions for participation in theelectoral process for all eligiblecandidates and voters, irrespectiveof gender, religion, ethnicity, politicalaffiliation, language, literacy or disability.• Genuine Elections — Elections must beheld for institutions which have authority,must be conducted in a credible manner,must present voters with real choicesbetween candidates for election, with theresults of elections representing the will ofthe people.• Periodic Elections — Elections must beheld frequently enough to ensure thatgovernmental authority continues toreflect the will of the people and that thereis regular opportunity for the voters tochange government.• Universal Suffrage — Legal andoperational limitations on access tocandidacy or the right to vote must beminimized and must not be discriminatoryin nature, except where such limitationsare reasonable or necessary. 19• Equal Suffrage — Voters should eachbe provided the same number of votesin each election being conducted andelectoral districts should be reasonablyequal in size so that each vote cast has asimilar weight.19 For example, on the basis of age, nationality,residence, mental incapacity or criminal conviction.• Secret Ballot — In order that votersbe able to freely express their electoralpreferences in the absence of intimidation,the ballot should be completed in privateand it must not be possible to link a voterto a voting preference.• Free Elections — The electoralenvironment must be such thatinformation on electoral contestants canbe made available to voters, informeddiscussion about electoral options cantake place, and voters are able to makeelectoral choices without intimidation.These political/electoral rights and standardsdo not operate in a vacuum. In fact politicalrights work in parallel with other humanrights and a healthy electoral environmentrelies on the realization of these broaderhuman rights. Human rights relevant tothe conduct of elections include the rightsto freedom of expression, 20 freedom ofinformation, 21 freedom of assembly, 22freedom of association, 23 freedom ofmovement, 24 to non-discrimination, 25 and toself-determination. 26 Transparency is also anessential component for a credible electoralprocess. The requirement for transparency isderived in part from some of the human andpolitical rights standards outlined above. 27It is also based on other internationalstandards, such as anti-corruption standards,20 Article 19 of the ICCPR.21 Article 19 of the ICCPR.22 Article 21 of the ICCPR.23 Article 22 of the ICCPR.24 Article 12 of the ICCPR.25 Article 2 of the ICCPR.26 Article 1 of the ICCPR.27 For example, the right to information, that electionsare credible (genuine) and that elections areconducted in a fair manner.18Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL STANDARDSwhich require public affairs to be conductedin a transparent manner. 28The international electoral standards outlinedabove are equally relevant for the use oftechnologies to assist the processes of votingand counting, as clearly stated in the Councilof Europe’s 2004 Recommendation on Legal,Operational and Technical Standards forE-voting, which states:“e-voting shall respect all the principles ofdemocratic elections and referendums.” 29Increasingly so, the use of new technologiesfor voting and counting are fundamentallychanging the way these components ofthe electoral process are conducted. Asa result, the use of technologies for votingand counting is also challenging this body ofinternational electoral standards.Some of these standards are no longeradequate to deal with electronic voting andcounting technologies. Other technologyrelated operations are not covered at all bythe existing set of standards. For example,it is clear that the use of electronic votingand counting technologies will have littleor no impact on the right to freedom ofmovement or freedom of association.However, other standards such as thesecrecy of the vote or the fairness of theelectoral process may be significantlyimpacted by the use of such technologies.28 See the United Nations Convention AgainstCorruption, especially articles 5, 7, 9, 10 and 13.29 Council of Europe (2004), p. 7.As a result, there have been initiatives inrecent years to evolve these internationalelectoral standards in order to copewith the challenges of using voting andcounting technologies. 30 The Council ofEurope’s 2004 Recommendation on Legal,Operational and Technical Standards forE-voting 31 did much to set the agendafor this adaption of existing standardsfor electronic voting and countingtechnologies. The CoE has followed up thisrecommendation with the publication of anE-voting Handbook 32 presenting guidelinesfor implementing e-enabled electionsand soon to be published guidelines oncertification and transparency for e-enabledelections. 33 In 2006 the EuropeanCommission also published a reporttitled Methodological Guide to ElectoralAssistance which covers support for theintroduction of election technologies,including electronic voting and counting30 It is worth noting that a number of nationalstandards have been developed to guide the use ofelectronic voting and counting technologies, suchas the US Election Assistance Commission’s (2005)Voluntary Voting System Guidelines (http://www.eac.gov/testing_and_certification/voluntary_voting_system_guidelines.aspx - last accessed on 30January 2011). However, these standards are onlynational standards and do not entail internationalobligations on other states. The sources referencedin the discussion on emerging standards all relateto international organization’s commitments orguidance to their members states, or internationalNGOs which are influential in the area of establishingelectoral standards.31 Council of Europe (2004).32 Caarls, S. (2010) E-voting Handbook: Key steps inthe implementation of e-enabled elections, Council ofEurope Publishing: Strasbourg.33 Council of Europe (forthcoming) Certificationof e-voting systems: Guidelines for developingprocesses that confirm compliance with prescribedrequirements and standards and Council of Europe(forthcoming) Guidelines transparency of e-enabledelections both drafted by the Council of Europe’sDirectorate of Democratic Institutions, “GoodGovernance in the Information Society” Project.International Foundation for Electoral Systems 19

INTERNATIONAL ELECTORAL STANDARDStechnologies and the standards that mightbe applicable in their use. 34The OSCE’s Office for DemocraticInstitutions and Human Rights, 35 theOrganization of American States, 36the Carter Center 37 and the NationalDemocratic Institute for International Affairs(NDI) 38 have also approached the issueof standards for electronic voting andcounting technologies from the perspectiveof observing elections in which thesetechnologies are used. Elections usingelectronic voting and counting technologiesare inherently less transparent than paperbased elections, as electronic events takeplace which are not possible to observewith the naked eye. 39 This makes it moredifficult to determine the credibility of the34 European Commission (2006) Methodological Guideto Electoral Assistance, see http://ec.europa.eu/europeaid/multimedia/publications/documents/thematic/ec_methodological_guide_on_electoral_assistance_en.pdf (last accessed on 31 January2011).35 OSCE (2005) Challenges of Election Technologiesand Procedures: Final Report, SupplementaryHuman Dimension Meeting, PC.SHDM.GAL/5/05;OSCE (2008) OSCE/ODIHR Discussion Paper inPreparation of Guidelines for the Observation ofElectronic Elections, ODIHR.GAL/73/08.36 OAS (2010) Observing the Use of ElectoralTechnologies: A Manual for OAS ElectoralObservation Missions, General Secretariat of theOrganization of American States (GS/OAS), seewww.oas.org/es/sap/docs/Technology%20English-FINAL-4-27-10.pdf (last accessed on 27 January2011).37 The Carter Center (2007) Developing a Methodologyfor Observing Electronic Voting, see http://www.cartercenter.org/documents/elec_voting_oct11_07.pdf (last accessed on 30 January 2011).38 Pran, V. and Merloe, P. (2007) Monitoring ElectronicTechnologies in Electoral Processes: An NDI Guidefor Political Parties and Civic Organizations, NationalDemocratic Institute for International Affairs, seehttp://www.ndi.org/files/2267_elections_manuals_monitoringtech-preface_0.pdf (last accessed on 2February 2011).39 OSCE (2008), p. 2.“Leading experts in thefield of e-voting argue thatthe lack of transparencywith electronic voting andcounting systems is thegreatest challenge facingthe implementation of suchtechnologies.”electoral process and whether any fraud ormistakes have taken place in their conduct.In fact leading experts in the field ofe-voting argue that the lack of transparencywith electronic voting and countingsystems is the greatest challenge facing theimplementation of such technologies. 40As a result, the use of electronic votingand counting technologies has presentedparticular problems for organizationsattempting to observe and evaluate theconduct of elections. Publications by theseleading election observation organizations areconsequently highly relevant to the debate onemerging standards for the use of electronicvoting and counting technologies.In analysing these important publications itis clear that some trends are emerging inthe recommendations being made by all ofthese organizations about the conduct ofelections using electronic voting and countingtechnologies. Common themes can be seenin the following areas:40 Krimmer, R. (Ed.) (2006) Electronic Voting 2006:Overview of Proceedings of 2 nd InternationalWorkshop, co-organised by the Council of Europe,ESF-TED, IFIP WG8.6 and E-Voting.CC.20Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL STANDARDS• Transparency — Transparency is relatedto many of the more specific emergingstandards below, but is importantenough to merit discussion separately.Transparency is a general electoralstandard, but one which is particularlychallenged by the use of electronicvoting and counting technologies.Special focus needs to be placed onthe realization of transparency whileusing these technologies. This meansthat as much of the operation of theprocess using electronic voting andcounting technologies is transparent orobservable. 41 However, access should beprovided for observers in a manner thatdoes not obstruct the electoral process. 42• Public Confidence — Closely related toand relying heavily upon transparency,is the requirement that votersunderstand and have confidence in theelectronic voting or counting technologybeing used. 43 Public confidencerequires that stakeholders are involvedin the introduction of electronicvoting and counting technologies, 44are provided information so theyunderstand the technologies beingused, 45 simulations of the systems takeplace 46 and voters are informed wellin advance about the introduction andwhat is required to participate. 4741 Council of Europe (2004) Recommendations 23 and56; OAS (2010) p.28.42 Pran and Merloe (2007) p. 62.43 Council of Europe (2004) Recommendation 20.44 The Carter Center (2007) p. 8.45 Council of Europe (2004) Recommendation 21; OAS(2010) p.20.46 Pran and Merloe (2007) p. 59.47 Council of Europe (2004) Recommendation 38;OSCE (2008) p. 14.• Usability — Electronic voting andcounting technologies must be easy tounderstand and use for as many votersas possible. 48 Users (voters) should beinvolved in the design of electronic votingand counting technologies 49 and in publictesting. 50 Furthermore, these electronicvoting and counting technologiesmust try to maximize the accessibilityof the voting system for persons withdisabilities 51 and afford voters thepossibility to stop and cancel their votebefore confirmation of their choice. 52• System Certification — Electronicvoting and counting technologiesmust be certified by an independentbody before use and periodicallythereafter. This ensures the systemcontinues to meet the requirements ofthe electoral jurisdiction as well as thetechnical specifications for the system.Furthermore, the certification processshould be conducted in a transparentmanner providing electoral stakeholdersaccess to information on the process. 53• System Testing — Any electronicvoting or counting system should besubjected to a comprehensive range oftesting 54 before it is approved for use48 Council of Europe (2004) Recommendation 1; OSCE(2008) p. 13; OAS (2010) p.20.49 Council of Europe (2004) Recommendation 62.50 OSCE (2008) p. 14.51 Council of Europe (2004) Recommendation 3; OSCE(2008) p. 13; OAS (2010) p.20; Pran and Merloe(2007) p. 76.52 Council of Europe (2004) Recommendation 14;OSCE (2008) p. 13-14.53 Council of Europe (2004) Recommendations 24 and25; OSCE (2008) p. 22; Pran and Merloe (2007) p.65-66 and 72; Carter Center (2007) p. 7.54 A full range of tests are described later in this guide.International Foundation for Electoral Systems 21

INTERNATIONAL ELECTORAL STANDARDSby an EMB, 55 This testing should takeplace transparently and with access forpolitical actors. 56• System Security — The opportunities forsystematic manipulation of the resultsmean that system security needs tobe taken extremely seriously. Securitymeasures need to be taken to ensurethat data cannot be lost in the event ofbreakdown, only authorized voters canuse an electronic voting or countingsystem, system configuration and resultsgenerated can be authenticated andonly authorized persons are allowedto access electronic voting, countingand results management functionality. 57Attempts to hack into electronic votingand counting machines or the electionmanagement system into which resultsare received, need to be detected,reported and protected against. 58• Audit and Recount — Electronic votingand counting technologies must beauditable 59 so it is possible to determinewhether they operated correctly. It mustbe possible to use an electronic voting orcounting system to conduct a recount. 60Such recounts must involve meaningfulmanual recounts of ballots cast55 Council of Europe (2004) Recommendation 31;OSCE (2008) p. 22; OAS (2010) p.20; Carter Center(2007) p. 7.56 OAS (2010) p.28; Pran and Merloe (2007) p. 67.57 Council of Europe (2004) Recommendations 32-33and 77-99; Carter Center (2007) p. 7-8; OAS (2010)p.19-20.58 OSCE (2008) p.12.59 Council of Europe (2004) Recommendation 59;OSCE (2008) p. 7; OAS (2010) p.20.60 Council of Europe (2004) Recommendation 26;OSCE (2008) p. 7.electronically 61 and not merely a repetitionof the electronic result already provided. 62• Voter-Verified Audit Trail — In addition tothe above requirements for auditabilityin any electronic voting or countingsystem, it must also be possible toassure voters that their votes are beingcounted as cast 63 while also ensuringthat the secrecy of the vote is notcompromised. 64 This requires thatelectronic voting systems 65 create anaudit trail which is verifiable. It shouldprovide the voter with a token/codewith which to perform the verificationexternally and not show the way inwhich the vote was cast. The mostcommon solution to this for in-personelectronic voting machines is through theproduction of a VVPAT, and this solutionis emerging as a standard in this regard. 66It should be noted that this VVPATsolution is not appropriate for remoteelectronic voting which uses electronicvoting machines (e.g. internet voting, textmessage voting etc.) as there would benothing to stop a voter from removing61 OSCE (2008) p. 19.62 Pran and Merloe (2007) p. 78.63 Carter Center (2007) p. 7.64 OSCE (2008) p. 18.65 Electronic counting machines have a natural voterverified audit trail in the paper ballot which wascompleted by the voter.66 OSCE (2008) p. 8 and 23; Pran and Merloe (2007)p. 72 and 75. Although it must be said that a voterverified paper audit trail is not the only way in whichthis can be achieved. In Belgium for example, the voteis stored on a magnetic card which can be verifiedon other voting machines before being placed in theballot box. This Belgian system is creates a voterverifiable audit trail without the use of paper. This isnot to say that the Belgian system is better or worsethan the VVPAT solution, merely to indicate that theremay be other non-paper methods of achieving thestandard.22Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL STANDARDS“Whenever electronicvoting and countingtechnologies are introducedthey should be done soin an incremental mannerand should start with lessimportant elections.”the paper record of the vote, making votebuying and voter coercion possible. 67• Mandatory Audit of Results — Theexistence of an audit trail for electronicvoting and counting systems achieve littleif it is not used to verify that the electronicresults and the audit trail deliver thesame result. Doing so also serves tobuild public confidence in the operationof the electronic voting or countingtechnologies. A mandatory audit of theresults generated by electronic voting orcounting technologies should be requiredby law and take place for a statisticallysignificant random sample of ballots. 68• Secrecy of the Ballot — The secrecyrequirement is not a new standard butit is one that is made more difficultby electronic voting and countingtechnologies. This is especially the casefor remote electronic voting systemswhere voters have to first identifythemselves and vote electronicallyusing the same interface. The useof electronic voting and countingtechnologies must comply with the needfor secrecy of the ballot. 69• Incremental Implementation —Whenever electronic voting and countingtechnologies are introduced theyshould be done so in an incrementalmanner and should start with lessimportant elections. This will allow publicunderstanding and trust to develop in thenew system, and provide time to dealwith problems and resistance. 70It is far too early at this stage to say thatinternational standards have completedtheir evolution in order to adapt to thechallenges posed by electronic voting andcounting technologies. Nevertheless, thetrends that can be seen in these emergingelectoral standards for the use of electronicvoting and counting technologies should becarefully considered as any new technologyis assessed.67 In fact, one of the greatest challenges facingremote e-voting remains the establishment of a voteverification mechanism for remote voters in an easilyunderstandable way which does not also provide away to violate the secrecy of the vote. There are somesolutions which provide codes to voters which can bechecked to see that the vote is included in the count,but nothing that can prove the value of the verifiedvote without relying on complicated mathematicalproofs which the average voter would have to trustjust as much as the operation of an electronic votingmachine.68 Council of Europe (2010) p.12; OSCE (2008) p. 18;Pran and Merloe (2007) p. 64 and 79.69 Council of Europe (2004) Recommendations 16-19;OSCE (2008) p. 11-12; Carter Center (2007) p. 9;OAS (2010) p. 19.70 OSCE (2008) p. 23; Carter Center (2007) p. 2.International Foundation for Electoral Systems 23

24Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

Decision in PrincipleThe decision to pursue new voting or counting technology must be made witha clear understanding of the needs being addressed and legal requirementsgoverning the voting process.This section outlines how to accomplish the first stage in the feasibility study process, the‘decision in principle.’ This critical stage aims to identify the objectives that are soughtthrough the introduction of new technology before measuring available technologies againstthese objectives. Establishing this foundation, the agenda for change, first and foremost willdo much to ensure that a well considered decision is initially taken as to whether electronicvoting and counting technologies can meet the requirements of the elections in question.The issue of cost will also be addressed in this stage. This issue determines whether thetechnology is feasible from a financial perspective and whether the benefits to be obtainedfrom the technology are sufficient to justify additional costs.All the components identified in this stage are seen as important in reaching a decision inprinciple on the feasibility of electronic voting and counting technologies. Other issues, specificto the electoral context, may be included for consideration. There is logic to the order inwhich these components are listed. The suggestion is that this order be roughly maintainedwhile implementing this stage of the feasibility study. Components later in this stage are moreproductive if preceded by the earlier ones. However, since this is a guide, the components of thedecision in principle should be adapted to the specific electoral requirements being considered.The Mandate of the StudyIt is critical at the outset of the feasibility study that the mandate of the study is clearlydefined. This mandate should be defined by the authority which initiated the feasibility study.International Foundation for Electoral Systems 25

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSThe mandate should identify the followingparameters for the study:to a Committee that does not entirely consistof EMB staff.• Purpose — The objectives the studyintends to meet need to be clearlyidentified. For example, is it to studythe potential use of electronic voting orelectronic counting, or both? Should italso assess the possibility for remotevoting — kiosk voting, Internet voting,mobile phone voting, etc.? Shouldany voting machine also be capableof identifying voters at the point ofpolling? What levels of elections wouldneed to be conducted using thesetechnologies? Will they be presidential,parliamentary, provincial, regional,local, etc.? These parameters will havesignificant impact on the conduct ofthe feasibility study and should be clearfrom the beginning.• Feasibility Study ProjectOrganization — Management of thefeasibility study will need to be entrustedto an organizational unit which plans andoversees different components of thestudy in an objective and impartial manner.Often a Feasibility Study Committee will beestablished for this purpose.The Feasibility Study Committee need notbe from the same institution or office whichprovides the mandate for the study. Forexample, a government institution couldrequest an EMB to investigate the feasibilityof using electronic voting and countingtechnologies. Even if an EMB itself decidesto investigate the suitability of using suchtechnology it may decide to entrust the taskIn fact, it is recommended to includemembership on the Feasibility StudyCommittee from a wider spectrum ofstakeholders 71 than from only one institution.Including multiple stakeholders is anadvantage because these technologiesstraddle the boundaries between legal,technical, social and political considerations.Feasibility Study Committee membersfrom the EMB are essential since they willimplement any solution agreed upon andhave a unique perspective on the possibilityof implementing voting and countingtechnologies. Information Technology expertswill also be required to properly assess thetechnological aspects of electronic voting andcounting technologies, especially securityand integrity aspects. Such experts may bepresent in the EMB. Government stakeholdersmay also be included in a Feasibility StudyCommittee since the government may haveto pay for any solution and pass legislationrequired to make the use of electronic votingand counting technologies possible.Other stakeholders to consider for inclusionin the Committee are: political partyrepresentatives, election related civil societyorganizations (e.g., domestic observerorganizations), organizations providing71 In fact the Council of Europe (2004) recommendationin e-voting recommends that users be involved in thedesign of the e-voting system and test the ease ofuse of the system at each main point in the process– recommendation 62. This does not mean thatthey have to be on Feasibility Study Committee, butdepending on how inclusive this Committee is it couldbe considered.26Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEelection technical assistance to the EMB,technology institutes and parliamentarians.A balance will need to be found betweenincluding stakeholders in the FeasibilityStudy Committee process and theeffectiveness of the Committee. Too large aCommittee may prove too cumbersome tobe effective. As recommended below, thereshould also be other opportunities to consultwith key stakeholders during the course ofthe feasibility study so the establishmentof a smaller, technical Feasibility StudyCommittee to manage the study does notpreclude an appropriate level of consultationwith key stakeholders.It is recommended that the same FeasibilityStudy Committee continue on to additionalstages which are conducted as part of thefeasibility process beyond the initial ‘decisionin principle’ stage. This provides consistencythroughout the process and ensuresinstitutional memory is maintained.tested, legislation amended, proceduresdeveloped, training and voter educationdelivered, post-pilot consultationsconducted and follow-on pilot projectsimplemented.• Format of Report andRecommendations — The mandateof the feasibility study should alsoindicate the recipient and the format ofthe report from the Committee on thedecision in principle. The report may berequired to provide recommendationson whether to proceed with pilotingelectronic technologies, on the mostappropriate technology, specificationsfor the technologies recommended, aplan and timeline for proceeding withpilot testing, the budget for pilotingand full adoption of the recommendedtechnology, etc. (Annex 2 has a sampletable of contents for the report for theFeasibility Study Committee report onthe decision in principle).• Timeline — An indication shouldbe provided to the Feasibility StudyCommittee as to how long it should bebefore they report back to the mandatingauthority on their findings. A suitableamount of time should be provided forthe study. A minimum of six months isrequired for a suitably comprehensivedecision in principle to be reached(Annex 1 shows a sample timeline forthe decision in principle stage). The laterstages of the feasibility study could takeyears to complete as electronic voting andcounting technology specifications aredeveloped, pilot machines procured andThe Feasibility StudyCommitteeThe use of electronic voting and countingtechnologies is a very technical issue andthe Feasibility Study Committee shouldbe well briefed on the kinds of issues andchallenges they need to consider, and bebriefed on recent developments in the field.A briefing pack of relevant electronic votingand counting technology materials shouldbe developed and provided to all membersof the Feasibility Study Committee — ideallybefore the first meeting of the Committee. IfInternational Foundation for Electoral Systems 27

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSit’s not possible to deliver the materials beforethe first meeting, then they should be ready atthe time of the first meeting (Annex 3 providesa sample briefing pack). In addition to writtenbriefing materials it is recommended toreceive verbal briefings from an organizationor organizations which have developed anexpertise in this field. Such organizationsinclude the OSCE, European Commission(EC), CoE, Organization for American States(OAS), IFES, NDI and the Carter Center, ifthey are active in the country in question.The first meeting of the Feasibility StudyCommittee might consider the followingagenda issues:• Review of the Mandate of the FeasibilityStudy Committee — It will be importantfor the Feasibility Study Committeeto review and fully understand themandate provided. A full understandingof the scope and limitations of thismandate will ensure that the FeasibilityStudy Committee remains focused onthe issues it has been requested toinvestigate. If the Committee determinedthat its mandate is not clear or itneeded to be extended, it might seekclarification/extension of its mandatefrom the authorizing institution.• Discussion of Key Challenges Relatedto the Introduction of Electronic Votingand Counting Technologies — It willalso be important for the FeasibilityStudy Committee to fully understand thepossible advantages and disadvantagesassociated with the use of electronicvoting and counting technologies atthe beginning of the feasibility study.The benefits and challenges listed inthe introduction to this guide can serveas a starting point. It may be relevantfor the Feasibility Study Committee tounderstand which of these issues arelikely to be relevant for their study.Briefing materials provided to theFeasibility Study Committee might alsobe discussed at this point, especiallyto identify other studies or experienceswhich might be particularly relevant. Thisagenda point provides an opportunityfor any international organization withexperience in the field of electronicvoting and counting technologies toprovide its advice and guidance to theFeasibility Study Committee, especially ifsuch organizations are already providingtechnical assistance to the EMB.• Overview of the Work of the FeasibilityStudy Committee — In line with thedifferent stages of a feasibility studyoutlined in the introduction, the FeasibilityStudy Committee should first focus onreaching a decision in principle as towhether electronic voting and countingtechnologies would be feasible for theelections being considered. The differentcomponents required to make thisdecision in principle should be discussedin detail so there is full understanding ofthe tasks involved.• Division of Responsibilities andEstablishment of Working Groups —There are many varied componentsinvolved in the decision in principle28Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEstage of the feasibility study and willmake sense to divide these componentsand allocate responsibility to severalworking groups or sub-committees(referred to as ‘working groups’hereafter). Again a clear mandate andmembership should be provided to eachworking group established.The membership of these working groupsmay extend beyond the membershipof the Feasibility Study Committee ifspecialized skills (e.g. IT or legal skills)would be useful in the working group andare not available on the Committee.One of the first tasks for each workinggroup is to develop a plan for the conductof its specific work and provide this tothe Feasibility Study Committee. TheFeasibility Study Committee may definethe timeframe for the conduct of workinggroup tasks when it establishes them.“There may be a lotof political interest in thedeliberations of the FeasibilityStudy Committee andany working groups if theCommittee is establishedthrough a governmental orparliamentary process.”• Openness of the Process — TheFeasibility Study Committee will need todecide the extent to which its work willbe open to observation, if not specifiedin the mandate. There may be a lot ofpolitical interest in the deliberations ofthe Feasibility Study Committee andany working groups if the Committee isestablished through a governmental orparliamentary process. The FeasibilityStudy Committee will need to determinewhether its meetings, and those of itsworking groups, are open to observers orclosed. If observers are allowed, then whois allowed to observe and mechanismsfor controlled and fair access to meetingsneed to be specified. Decisions will haveto be made as to whether materials,workplans and interim reports are madeavailable to interested parties.Since steps identified in this guideprovide for significant levels ofconsultation from stakeholders, suchopen access and transparency tothe Committee and working groupsmay not be required. However, if thefeasibility study is in the public orpolitical spotlight there may be greatinterest in the work of the FeasibilityStudy Committee. Providing this accessmay help increase the credibility of theCommittee and its recommendations.• Schedule of Meetings and WorkingGroup Updates — The FeasibilityStudy Committee should establish aregular meeting schedule for itself, andrequire working groups report back atthe first meeting with a workplan andtimeline for their activities. Establishinga regular meeting schedule will assist intransparency in the process if meetingsare open for observers.International Foundation for Electoral Systems 29

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSIt is also worth noting that the work of theFeasibility Study Committee, and the workinggroups, may be facilitated by establishingsupport services, similar to a ‘secretariat’.This secretariat would support the workof the Feasibility Study Committee bypreparing meeting documents and briefingpacks for members, assisting in draftingagendas, informing members of meetingarrangements, drafting minutes from themeetings (where necessary) and drafting thereport of the Committee.Vendor RelationsA dialogue with vendors is an essential part ofany feasibility study. Information is required fromthe vendors about the technologies in order tounderstand the products which are currentlyavailable on the market. The information initiallyprovided by vendors may leave many questionsunanswered. This will require further clarificationfrom the vendors. Through the course of thefeasibility study the requirements which thesetechnologies are being measured againstmay evolve, necessitating follow on requeststo vendors to see if they can still meet thesechanging requirements.A potential country-wide implementation ofelectronic voting or counting technologiesis a profitable exercise for vendors, andone that is bound to increase a vendor’sreputation and profile. Therefore, theCommittee will need to ensure that it dealsfairly and equally with all electronic votingand counting technology vendors. TheCommittee should protect itself from anyallegations of favoritism and inappropriateconduct/contact with vendors.Many countries have clear regulationsdefining the way in which public institutionscan communicate with companies which are,or may be, likely to submit tender proposals.The approach to vendor relations providedhere should in no way be seen as suggestingthat these regulations be ignored. TheFeasibility Study Committee needs to ensureit understands any procurement and vendorrelations regulations before it determinesits communication strategy with vendors.Within the limitations imposed by thesenational regulations for public institutions, theEuropean Commission and UNDP’s recentpublication on procurement proceduresfor election technologies 72 can be used toguide relations with vendors and the broaderprocurement process.To the extent possible it is recommendedthat contacts with vendors be establishedearly on in the process so vendors canhave time to consider and respond to theFeasibility Study Committee’s requestsfor information. It is suggested that onepoint of contact be established for theCommittee’s contacts with vendors. Thispoint of contact (POC) should, to the extentpossible, ensure that the same informationis provided to all vendors. The POC mayconsider having the Committee approve allcommunications with vendors.72 Joint EC-UNDP Task Force on Electoral Assistance(2010) Procurement Aspects of Introducing ICTSolutions in Electoral Processes: The Specific Caseof Voter Registration, see http://aceproject.org/eroen/misc/procurement-aspects-of-introducing-ictssolutions(last accessed 24 January 2011).30Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEFeasibility Study CommitteeWorking GroupsA comprehensive feasibility study needsto investigate the use of electronic votingand counting technologies from a range ofperspectives and deal with technical andcomplex issues. A feasibility study requiresa lot of work. Many of the issues beingconsidered may require, or benefit from, theinput of specialized personnel (e.g., lawyers,IT experts and communications specialists).Therefore, it may make sense to divide thework of the Feasibility Study Committee intoseveral working groups where specializedpersonnel can be called.The number and mandate of working groupswill be very context specific. It will dependon what key issues are most relevant tothe particular electoral environment at thattime. The suggested list below representsthe minimum key issues that shouldbe addressed by the Feasibility StudyCommittee. Separate working groups neednot be created to deal with each of theseissues. It may be possible for one workinggroup to cover several issues.Many of the issues considered by workinggroups are very technical in nature and willbenefit from technical advice of expertsin relevant fields. Any EMB would be welladvised to secure the services, either directlyor through a technical assistance provider,of an expert, or experts, in this field toadvise them on these issues and guide themthrough possible pitfalls.Issue 1: Assessment of the CurrentSystem of Voting and Counting — A keycomponent of any feasibility study on the useof electronic voting and counting technologieswill be to determine what a change from thecurrent system to one with such technologieswill achieve. Only by fully defining this will it bepossible to determine if the available solutionscan meet these requirements for change andwhether it is feasible to implement them forthe elections in question.Reaching a conclusion on this issue will requirethe following questions to be answered:• What are the strengths of the currentballoting system?• What are the weaknesses of the currentballoting system?• Can any or all of the weaknessesidentified in the current system beaddressed through reform of the existingballoting system?• If so, what would be required to addressthese weaknesses in the current systemof balloting?• What improvements are expected fromimplementing change to the currentsystem of balloting?• What desired improvements cannot beeasily resolved through reform of thecurrent system?The working group should write up itsfindings and submit a report to the FeasibilityInternational Foundation for Electoral Systems 31

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDS32Study Committee. The report of this workinggroup constitutes an important component ofthe overall feasibility study, helping define theagenda for change — the objectives soughtthrough the introduction of electronic votingand counting technologies.Issue 2: Assessment of the Advantagesand Disadvantages Offered by Voting andCounting Technologies — Even if a significantagenda for change is identified by the workinggroup considering issue 1 above, usingelectronic voting and counting technologiesmay not be the solution. It is also important torecognize that using such technology presentsnew challenges to the conduct of elections.Consideration of this issue will consist of twoaspects, a general assessment in principleof what technology has to offer in terms ofelectronic voting and counting technologiesand an assessment of the solutions currentlyoffered by a range of vendors. In order to dothis, electronic voting and counting technologyvendors will need to be contacted and asked toprovide information on their current products.The following questions will need to beanswered by the working group addressingthis issue:• What are the advantages thatelectronic voting and countingtechnologies offer compared to thecurrent balloting systems?• What are the disadvantages ofusing electronic voting and countingtechnologies compared to the currentballoting systems?Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies• Are there external infrastructurerequirements and resourcerequirements within the EMB that wouldbe essential in implementing electronicvoting or counting technologies(e.g., electricity or communicationsrequirements, specialized skills in theEMB)? Do these infrastructure andresources currently exist? If not, whatis required to provide the necessaryinfrastructure and resources?• On the basis of the consideration ofthe above three issues, what are therequirements that any new electronicvoting or counting technology would needto fulfill in order to meet the objectives ofholding the elections in question?• What specific challenges would the EMBface in implementing electronic voting andcounting technologies, including trainingof staff, voter education, cultural sensitivity,stakeholder trust, specialized staff skillsrequired, physical and data security,storage and maintenance requirementsand preparation prior to elections?Consideration of these questions forms acritical component of any feasibility study.It is essential that sufficient thought is givento these issues as failure to do so could“One possible result of afeasibility study may be thatno suitable electronic votingand counting technologyproducts are foundwhich meet the electoralrequirements in question.”

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEfundamentally affect the success or failureof any technology project. Of particularimportance is the development of a setof requirements that the solution wouldbe required to meet. If this is not properlydefined then the solution recommended bythe feasibility study may not be appropriatefor the electoral process.The development of this set of requirementsmay also be of importance beyond thescope of the feasibility study. One possibleresult of a feasibility study may be that nosuitable electronic voting and countingtechnology products are found which meetthe electoral requirements in question. Ifthis is the case, the set of requirementsidentified by addressing this issue willremain valid and can be used in the futurewhen new products are developed whichmight better meet these requirements.Issue 3: Review of IT Security Aspects —System security is an incredibly importantfeature of electronic voting and countingtechnologies. These technologies areinherently less transparent than the use ofpaper ballots, where all steps in voting andcounting are observable. If an electronicvoting or counting system is to be properlytrusted by electoral stakeholders it isimportant that the security challengespresented by the use of the technology areunderstood. Mechanisms should be in placeto mitigate these security challenges and anysecurity breaches should be easily identified.There are a number of questions that needto be considered by the working group onthis issue:• Will the source code for the electronicvoting or counting technology be opensource or not?• How will the source code be tested andcertified?• How will it be verified that the sourcecode used for the conduct of electionsis the same as the one tested andcertified?• What mechanisms are in place toensure that the new system is protectedagainst tampering and that it can beeasily determined when it has beentampered with?• If results are electronically transmittedfrom electronic voting or countingmachines to a regional or centraltabulation facility, how will the resultsbe encrypted to ensure there is nounauthorized access to the results?• How will results transmitted electronically,to a regional or central tabulation facility,be verified on receipt to ensure that theyare legitimate and they are submittedfrom an authorized officer or location?The working group addressing thesetechnical issues will need to make sure thatit is able to clearly articulate the results ofthe discussions around these issues to theFeasibility Study Committee. This will bevery important in order to provide technicalrequirements to the working group dealingwith issue 2 above. It will also help definethe technical components of any laterInternational Foundation for Electoral Systems 33

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSprocurement process and ensure any legalamendments properly address the technicalissues discussed and agreed upon.The issue of open source code is worthy offurther discussion. Whether source code isopen source or not is a significant issue indebates about the security and transparencyof electronic voting and counting systems.Source code is the set of instructions thatdetermines how the system functions.Traditionally, with electronic voting andcounting systems developed by commercialorganizations this source code has beendeemed to be proprietary in nature, exclusivelyowned by the supplier and not by the EMB.While an EMB may procure the right to use thesoftware, and potentially the right to analyzethe source code, there is no right to modify orfurther distribute the source code. 73Alternatively, the source code can be madeopen source. Open source code is publicallyavailable to all interested parties. Anyone cananalyze the code and determine whether itaccurately functions to record the intentionsof the voters, or identify mistakes in thecode. While the pool of people who canread and understand computer source codeis relatively limited (compared to the overallvoting population), the idea is that there areenough who can do so to provide a checkand balance to ensure the source codefunctions correctly. It is also expected thatpublishing the code makes the deliberateinclusion of malicious code less likely. Usingopen source code is a way to increase trustin the use of electronic voting and counting73 See Council of Europe (2010) E-Voting Handbook,p. 16.technologies. It can significantly reduce thecosts involved in developing them. 74This is not a simple decision for an EMB totake, especially as requiring open sourcecode may dissuade some suppliers fromsubmitting proposals. Suppliers maysee the source code as their intellectualproperty and the result of many years ofdevelopment resources.One alternative to open source or proprietarysoftware could be to allow a limited‘independent, expert group’ to have accessto the source code to review it before it isused for an election. Such a group wouldhave full access to the code, but would agreeto not disclose the source code. There arecomplications in this approach becausea proper report of findings is open andtransparent. The report may inadvertentlydisclose proprietary aspects of the sourcecode. On the other hand, a closed reviewand report which is not made available tostakeholders is not likely to generate thetrust and confidence in the process that thisreview is meant to achieve.The decision on the status of the sourcecode will need to be considered prior to thecommencement of the procurement process.It may be preferable to leave the final decisionon this open at this stage. For example, ifa decision was taken to pursue an opensource option then this may restrict many oreven eliminate all of the vendors who mightotherwise be interested in the project. Fornow, a preference might be identified and74 Ibid, p. 17.34Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEcompliance with this preference could be afactor taken into consideration during anysubsequent electronic voting and countingtechnology procurement process.made at the beginning of the process,followed up by the provision of additionalinformation when, for example, a detailedset of requirements have been developed.Issue 4: Determining TechnicalFeasibility — Once a set of requirementsfor a possible electronic voting or countingsolution has been defined it will need tobe determined whether products exist, orcould be developed, which meet theserequirements. A full consideration of this issueobviously requires information on currentproducts. This information should be providedby vendors of electronic voting and countingtechnologies. It will be important to ensurea wide range of vendors are contacted andgiven the opportunity to provide informationon their products. To do otherwise may resultin accusations of bias on the part of theFeasibility Study Committee.Once information has been receivedfrom a suitable number of vendors, eachrecommended product should be measuredto see the degree of compliance with the set ofrequirements. This analysis of electronic votingand counting technology products against therequirements will determine whether the use ofthese technologies for the elections in questionis technically feasible or not.If the result of this analysis is that noelectronic voting and counting technologyproducts are found which meet the set ofrequirements, and therefore the needs ofthe elections in question, then a number ofoptions are available:In fact, an ongoing dialogue with thesevendors may be relevant as the work ofthe feasibility study progresses. Informationinitially provided by vendors may well leadto a number of follow on questions fromthe working group. It will also be importantto inform vendors when a detailed set ofrequirements is developed so vendors canbetter understand the electoral needs andrespond to the requirements with targetedproduct solutions.Vendors may take some time to respondto a request for information from theFeasibility Study Committee; therefore,the request for information from vendorsshould be initiated early in the process.An initial request for information could be• Reconsideration of the Requirements —It may be that the standard is too highin the requirements developed for apossible electronic voting and countingtechnology solution. If one or more ofthe components of the requirements areconsistently not met by the productssuggested then these components of therequirements should be reconsidered.Are they really necessary? Could therequirements be softened in a way thatis acceptable so some electronic votingand counting products complied withthe requirements? The answer to both ofthese questions may well be “no,” but itis worth consideration.International Foundation for Electoral Systems 35

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSVoting and counting technology can be used in almost any election, but it shouldbe used only if it meets all the requirements of that specific election.• Contacting Additional Suppliers —Depending on the initial responsefrom suppliers for information on theirproducts, and the number of supplierscontacted, it might be worth increasingthe number of electronic voting andcounting technology vendors contactedor generating a greater level of responsefrom those originally contacted.Surveying a larger number of productsmight help to find ones which do meetthe requirements.• Development of New Products — Itis possible that the requirementsdeveloped for the feasibility study areunique; products may not have yetbeen developed. This does not meanthat products cannot be developed tomeet the requirements. The FeasibilityStudy Committee may decide to requestvendors to consider the possibility ofdeveloping new products to meet thecriteria. In fact vendors may be very willingto adapt software or even hardware toemerging requirements as part of theirbusiness development strategy.It may be that all these options fail to provideelectronic voting and counting technologyproducts which meet the requirementsidentified in the feasibility study. In this case,the Feasibility Study Committee wouldconclude that the electronic voting andcounting technologies on the market do notmeet the needs of the electoral situation.This is an important caveat for finding thatcurrent electronic voting and countingtechnologies is not feasible. Of course,almost any election can be conducted usingan electronic voting or counting solution,but it is not necessarily the case that such a36Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEsolution should be used. This would requirethat the electronic voting and countingtechnology meet all the requirements of theelection in question.Finding that using electronic voting andcounting technologies for elections is notfeasible is not a failure for the study. Infact, if the previous steps in the study areconducted comprehensively then the studywill lead to a well defined set of requirementsfor an appropriate electronic voting andcounting technology solution. This set ofrequirements will remain valid and can beused to re-assess, on a periodic basis, anynewly developed products.Issue 5: Cost Benefit Analysis — Shouldan electronic voting and counting solution,or solutions, be found which meet therequirements previously identified then afurther assessment will need to be madeas to whether the implementation of thesesolutions would, on balance, be beneficialand cost effective.There are two components to this analysis.Before the analysis can be conducteda limited number of electronic voting orcounting solutions will need to be selectedfor cost benefit analysis purposes, as theprocess is quite complex to conduct. Thebest electronic voting or counting solutionand the cheapest solution, which still meetsthe requirements, should be selected.Another electronic voting or counting solutionwhich is mid range in terms of cost and interms of meeting the requirements could alsobe selected.The first step is to identify the benefits thateach solution provides compared to thecurrent system of balloting. Similarly a listof disadvantages/challenges associatedwith each solution should be identified. Thecomparison of these two lists of advantagesand disadvantages of the different electronicvoting or counting solutions will show theoverall benefits of using each solution.There is no predefined formula involved inthis assessment of beneficiality. It could bethat there are many disadvantages involvedin using an electronic voting or countingsolution and only one benefit. However, thatbenefit could be of such critical importancethat it would still support the introduction ofelectronic voting or counting technologies.In addition, the importance attached toeach advantage and disadvantage willbe determined by the particular electoralcircumstance. Therefore this analysisof advantages versus disadvantages issomething that can be done in a Committeeformat, but is probably something thatshould be consulted on very widely amongstelectoral stakeholders to ensure there isconsensus on the recommendations resultingfrom this assessment.The second stage of this cost benefit analysisrequires a comprehensive cost analysis ofthe technology and a comparison of costsassociated with using this technology visà-visthe existing system of balloting andcounting — likely paper-based voting.A key component of this cost analysis isto recognize that the costs associatedwith using electronic voting or countingInternational Foundation for Electoral Systems 37

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDStechnologies should not be considered solelyon the basis of the initial investment and acomparison of this cost against the cost ofpaper balloting for the next election (if paperballoting is the current system). Electronicvoting and counting technologies are aninvestment which last for a long period oftime and will be available to use for manyelections. Therefore, the initial investmentneeds to be considered and compared to thecurrent system recognizing this is a long termperspective and there are additional costsassociated with maintaining and using thesetechnologies beyond the initial investment.This is achieved by calculating the costsassociated with each solution over the lifecycle of the electronic voting or countingtechnology, such that initial investment costsand ongoing costs are averaged out overthe number of elections that the electronicvoting or counting solution would last. Byadopting this approach an average cost perelection of using a system can be calculatedand compared to a similar calculation for theexisting system.This comparison needs to take intoconsideration the fixed costs associatedwith the existing and the electronic voting orcounting systems, as well as the variable costsassociated with each system. Fixed costsrelate to initial investments in the system whichwill not be repeated each time an election isconducted; it could include the following:• Paper Balloting--Cost of replacing damaged ballotboxes--Cost of replacing damaged votingbooths--Cost of storage of ballot boxes andvoting booths• Electronic Voting or Counting Technology--Cost of buying the electronic votingor counting technology--Cost of procuring central utilities forconfiguration and result management--Cost of buying stands/voting boothsfor the electronic voting machines--Cost of storage of electronic votingor counting technologies--Cost of repair/replacement ofbroken electronic voting or countinghardware--Cost of independent testing andcertification of the technologies--Cost of developing new proceduresand legislation to use the technologyIn some ways this comparison of fixed costsbetween the existing system and a proposedelectronic voting or counting system is unfairas the initial investment in the existing systemhas already been made (e.g., ballot boxesand voting booths will have been procured).Therefore, only the replacement costs of fixedassets associated with using the existingsystem, resulting from expected wear andtear and expected life expectancy of ballotboxes and voting booths that need to beconsidered for the existing system. Whereasfor the proposed electronic voting or countingsystem, the fixed costs associated withestablishing a completely new system willneed to be considered. While in principle thismeans that the comparison is a little unfair,38Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEit does represent the reality of the decisionbeing considered — the costs of continuingwith the existing system versus thoseassociated with a completely new system.Variable costs relate to the kinds of costs thatare repeated for each election that the systemis used and could include the following:• Paper Balloting--Cost of printing ballot papers--Cost of printing voter lists for thepolling station--Cost of transportation of materials(ballots, ballot boxes, votingbooths, etc.)--Cost of ballot box seals--Cost of voter marking ink--Cost of other polling materials(official stamps, ink pads, forms,envelopes, etc.)--Cost of polling staff required forpaper balloting--Costs of educating voters on paperballoting• Electronic Voting or Counting Technology--Cost of printing voter lists for thepolling station or uploading voter liststo electronic voting machines--Cost of transportation of materials(electronic voting or countingmachines, stands/voting booths, etc.)--Cost of replacement batteries forelectronic voting or counting machines--Cost of ballot papers for electroniccounting systems or paper for anyPaper Audit Trail with an electronicvoting machine solution--Cost of voter marking ink--Cost of other polling materials(voting machine activation keys,official stamps, ink pads, forms,envelopes, etc.)--Cost of polling staff with thenecessary skills to administer the useof the electronic voting or countingtechnology--Cost of any additional pollingstation infrastructure required bythe electronic voting or countingtechnology--Cost of any results data transmissionhardware (connectivity or portablestorage devices)--Cost of specialized staff/techniciansrequired to configure, test andsupport the electronic voting orcounting technology--Costs of managing the change insystem, including significant costs oftraining staff and educating voters onusing electronic voting or countingtechnologyThese fixed and variable costs varyfrom electoral environment to electoralenvironment. It is highly likely that the initialinvestment costs associated with usingelectronic voting or counting technologiesare much higher than those associatedwith paper balloting (even if a completenew investment in ballot boxes was beingconsidered). The variable costs associatedwith using an electronic voting system maybe much less than those associated withpaper balloting, if only because electronicvoting systems (but not electronic countingsystems) will not require paper ballots toInternational Foundation for Electoral Systems 39

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSbe printed. In most elections the printing ofballots represents a significant cost. 75The process of determining the costsassociated with paper balloting or usingelectronic voting or counting technologies iscomplex. It is difficult to isolate all the costsassociated with paper balloting and identifyall of the costs associated with a systemwhich has not been used before. Therefore,to some extent the cost analysis will be anexercise in best guessing such costs, but theexercise is nevertheless important.While estimating the fixed and variable costof the existing (e.g., paper) ballot system, theworking group should consider incorporatingthe cost of implementing any requiredimprovements to this system identified underissue 1 above. Under all circumstances,historical data related to the cost of theexisting system in past elections should beadjusted for likely future price inflation.Once all fixed and variable costs have beenidentified for paper balloting and the electronicsolutions being considered, the total costs ofusing the technology can be calculated overthe life expectancy of the electronic voting orcounting technology hardware.All of the elections that electronic votingor counting technologies will be used forover their lifetime need to be determined75 This assertion is somewhat supported by the findingsof the Election Commission of Pakistan’s (2010)Committee on the Use of Electronic Voting Machinesin Pakistan: Final Report and Recommendation, p.31. While the cost analysis in this report does notinclude all of the costs identified here, it does indicatea much lower variable cost level for using electronicvoting machines compared to paper balloting.and the total cost of using the technologiesfor these elections calculated. Thiscalculation will be the fixed costs plus thevariable costs per election multiplied by thenumber of elections over the lifetime of theelectronic voting or counting technology.From this the average cost of using theelectronic voting or counting technology perelection can be calculated.Total Costs of = Fixed Costs of TechnologyUsing Technology + (Variable Costs of Using Technologyx Number of Elections)Average Costs of = Total Costs of Using TechnologyUsing Technology ÷ Number of ElectionsA similar calculation will need to be made forcontinued use of the existing system in orderto generate a per election cost of continuingwith the existing system of balloting.These two costs then need to be comparedso that, given the life cycle of the electronicvoting or counting technology underconsideration, it can be determined what thedifference in cost will be between using thetechnology and continuing with the existingsystem of balloting. The difference in costsneeds to be weighed against the additionalbenefits provided by the electronic votingor counting solution and the disadvantagesassociated with it.Again, there is no formula with whichthis overall cost-benefit analysis can bemade. The process above is an attempt toclearly identify the costs and benefits (anddisadvantages) associated with the use ofelectronic voting or counting technologies. Insome cases this analysis will clearly indicate40Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEone decision or another. For example, if thebenefits of using electronic voting or countingtechnology are many and the additional perelection cost are small then there is a strongcase for the new technology.More likely the ‘balance sheet’ will be verymixed. There may be a significant additionalcost involved in using electronic voting orcounting technologies but some importantbenefits resulting as well as some potentialproblems. It will be up to the Feasibility StudyCommittee to decide whether the benefitsto be realized by using electronic voting orcounting technologies are sufficient to justifyany additional expenditure and make itsrecommendation accordingly.Issue 6: Institutional Capacity — A criticallyimportant issue for the working groups toconsider is whether the institutional capacityexists to implement electronic voting andcounting technologies. This issue does notonly relate to the EMB, but also to otherbodies which would support the conduct ofelections using these technologies.A number of key areas should be consideredin order to reach this assessment:• EMB Organizational Capacity — Themanagement of an electronic votingand counting technology project is anincredibly complex task, even if onlyfor a small pilot of the technology. TheEMB will need to coordinate a range oftasks to implement the project, includingprocurement of the technology, logistics,procedural development, training,voter education and IT configuration“If the benefits of usingelectronic voting or countingtechnology are many andthe additional per electioncost are small then there isa strong case for the newtechnology.”and support. This will require thatthe capacity exists within the EMB toprovide sufficient qualified resources forimplementation of the project.• Training Capacity — Implementationof electronic voting and countingtechnologies requires a significantrevision of the procedures for polling andcounting. Revised procedures, howevergood, will have little effect if they are notproperly communicated to all staff whowill implement them. This will require arobust training infrastructure, especiallyif it is required to deliver training to allpolling and counting staff.• Information Technology Capacity —Significant information technologyexpertise will be required at many pointsduring the implementation of electronicvoting and counting technology projects.The EMB will need to conduct full testingof any electronic voting or countingtechnology being used. The centralelection management system will needto be configured for the elections beingheld and the candidates/political partiesstanding for election. If electronic voting orcounting machines are being used thenthese machines, possibly tens or hundredsInternational Foundation for Electoral Systems 41

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSof thousands depending on the size of theelection, will also have to be configuredand tested. Finally, the EMB will need toprovide technical experts on Election Daywho can resolve technical problems whicharise while using the technologies. This willrequire skilled, and possibly large numbersof, IT staff within the EMB.• Polling and Counting Staff Capacity —Wherever new technologies areimplemented, at the polling station orlocal counting centers, the staff whowork in these polling or counting centerswill have to be sufficiently IT literate tooperate the electronic machines. Anassessment of the technical capabilitiesof these staff will be essential indetermining this level of capacity.• Voter Education Capacity — Voters willneed to be informed well in advanceabout the use of electronic voting andcounting technologies and how tointeract with the technology. This willrequire that effective mechanisms exist,either directly through the EMB or inpartnership with political parties and civilsociety, to communicate these votereducation messages.• Independent Certification Capacity —The independent certification of electronicvoting and counting technologies is a veryimportant aspect of building trust in thenew technologies, It will be covered laterin this guide. An assessment will need tobe made as to whether independent andtrusted technology organizations whichcould conduct this testing and certificationof electronic voting and countingtechnologies exist.Not only will such capacities be requiredto ensure that any electronic voting orcounting technology project is implementedsuccessfully, it will also be necessary toensure that the EMB remains in control ofthe process. Some electronic voting andcounting technology projects have seenvendors filling gaps in the institutionalcapacity of domestic institutions. Whilethis has often been done in the interest ofimplementing the project successfully, itrepresents an abdication of responsibilityon the part of the domestic institutionand creates an unhealthy dependency onthese vendors. It also indicates a lack ofsustainability in the use of the electronicvoting or counting technologies.It may be that in assessing institutionalcapacities required for successful electronicvoting and counting projects, some or allof the assessments may state that thecapacity does not exist. This will need to beadded into the overall consideration of thedecision in principle. However, a negativeassessment of the capacity on any of theseaspects of institutional capacity need not bean insurmountable obstacle. It may be thatthe capacity does not currently exist, butcould be developed by certain strategies.Where this is the case any insight intopossible strategies to develop the requiredcapacity will represent important additionalrecommendations from the working group.42Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEIssue 7: Legal Reform Issues — The finalissue for consideration concerns the possibilityfor using electronic voting or countingtechnologies under the existing electoral legalframework. It may well be that the existingelectoral legal framework makes referenceto physical ballot boxes and ballot box seals,to actual ballot papers and the ways inwhich ballots are counted and adjudicated.Obviously all of these processes occur withan electronic voting or counting machine. Theworking group needs to assess whether itwould still be in compliance with existing law.Therefore the electoral legal frameworkneeds to be reviewed to determine if it isin compliance with using electronic votingor counting technologies. It is highly likelythat if only paper balloting has been usedin the past then the laws will be written insuch a way as to preclude the use of thesetechnologies. The working group will need todo the following:• Identify the parts of the current legalframework for elections that need to beamended in order to allow an electronicvoting or counting system.• Propose the amendments that would berequired to allow the use of an electronicvoting or counting system.In addition to this, the working group dealingwith this issue may wish, and may be advised,to take a more comprehensive look at thelegislation governing elections and how itwould relate to the implementation of anelectronic voting or counting technology.Merely adapting the existing legislation soit does not preclude the use of voting orcounting technologies is not sufficient toproperly regulate the use of these technologies.The working group should also consider thefollowing issues and legal amendments:• The transparency mechanisms thatwould need to be implementedthrough the election law in relationto the use of electronic voting andcounting technologies, including accessto key components of the electionadministration process using thesetechnologies for observers.• Security mechanisms and safeguardswould need to be established inlegislation to ensure the accuracy andintegrity of elections using electronicvoting and counting technologies.• Legal requirements for initial andperiodic independent certification ofelectronic voting and counting systems.The institutions which are permitted toconduct this certification. The registrationprocess and requirements for certifyinginstitutions and consequences of noncertificationof electronic voting andcounting technologies.• The status of an election if themechanisms for producing an audit traildid not work (e.g., the printer did notwork or the machine ran out of paper).• The legal status of the electronic record ofvoting produced by an electronic votingor counting machine compared to theaudit trail record, and which record takesprecedence in the event they are different.International Foundation for Electoral Systems 43

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDS• If the audit process results in differentresults than generated electronically,is there a requirement for mandatoryaudits of electronic voting and countingmachines after the election? Whatis the scale of this mandatory auditprocess? What are the selectionmechanisms for the mandatory auditand possible consequences?• The mechanisms for challenging resultsgenerated using electronic voting andcounting technologies and instances inwhich a challenge against the result willlead to a manual recount of the audit trailfor the voting or counting machine.While conducting this task, a number ofpoints should be borne in mind. First,legislation should be amended to allow,not require, the use of electronic voting orcounting technologies. Changing legislationin this manner does not mean that electronicvoting or counting technologies have tobe used. Legislation should allow for thepossibility of using electronic voting andcounting technologies and paper ballotingin different locations at the same time. Thiswill likely be the case when the technology ispiloted or even if the technology is introducedin a phased manner. Second, the processof legal amendments may be a lengthy one,therefore, if legal changes are required inorder to use electronic voting or countingtechnologies then it is prudent to start theprocess as early as possible, based on thefindings of the working group.Study TripsThe Feasibility Study Committee mayconsider the possibility of conducting oneor more study trips to see other countrieswhich have used or are using electronicvoting or counting technologies. Study tripsshould take place to countries with similarelectoral circumstances to the one in whichthe feasibility study is being conducted.This does not mean that lessons cannotbe learned from very different electoralenvironments. It would also make senseto visit countries which are implementingtechnologies of interest to the FeasibilityStudy Committee. For example, it wouldprobably make little sense to visit a countryonly implementing internet voting if this wasnot an option being considered.Any study trip should meet with a rangeof stakeholders, including the EMB,the technology provider, political partyrepresentatives, civil society representatives,voting activists and domestic electionobservation organizations. The study tripshould seek to address the following issues:• Type of technologies that have been orare being used.• Process followed in taking a decision toadopt the technology.• Stakeholder opinions on the advantagesand disadvantages of these technologies.• Challenges presented by using thetechnologies, and the ways in whichthese challenges had been met.44Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLE• Country specific factors which led tothe success or failure of using thesetechnologies.Depending on the size of the Feasibility StudyCommittee it may not be possible for allmembers to participate in such a study trip.If this is the case then it will be important toensure that proper reporting of the study tripfindings and recommendations are made tothe Feasibility Study Committee and becomepart of the record of the proceedings of thefeasibility study.Vendor DemonstrationThere is only so much that can be revealedabout a system by reading technicalspecifications and marketing materials aboutelectronic voting or counting solutions. A fullerunderstanding can only be achieved by seeingelectronic voting or counting technologies inaction, initially through a demonstration. Such ademonstration is far superior to even seeing thetechnology being used for an actual election.This allows participants in the demonstration toask questions along the way and try to violatethe procedure or ‘break’ the machine.The demonstration environment allows for adetailed discussion between the FeasibilityStudy Committee and the vendors about theways in which their products work, or couldbe adapted to work. Again it is importantthat a wide range of vendors are invited topresent their products at the demonstrationso any perception of favoritism in the processis countered. Organizing such an event with asuitable range of vendors may take quite sometime to arrange, so it is important to initiatepreparations for the event well in advance.Participation at the vendor demonstrationevent needs to be carefully considered. Thevendor demonstration could be limited toonly the Feasibility Study Committee itself,or could be opened up to a wider groupfrom the EMB. It is recommended thatparticipation in any vendor demonstrationbe widened to include representatives frompolitical parties and civil society. Theseare important stakeholders in the electoralprocess; providing them access to the vendordemonstration will help their understandingof recommendations made by the FeasibilityStudy Committee. It also means thatconsultations held with these stakeholderscan take place from a more informed startingpoint. On the other hand, excluding thesegroups from the vendor demonstrationmight be helpful so expectations are notcreated. If included, the Feasibility StudyCommittee needs to be very clear withstakeholders about the purpose of its work,the demonstration and that it represents aninitial investigation into the feasibility of usingelectronic voting and counting technologies.The timing of the vendor demonstration in theprocess of the feasibility study is important.If held too early in the process, the FeasibilityStudy Committee will not be sufficientlyinformed about the relevant issues relatedto the use of electronic voting and countingtechnologies. The Feasibility Study Committeecan make the most of its face-to-faceinteraction with vendors if it engages with thevendors from an informed position. Waiting untilthe work of the Feasibility Study Committeeand its working groups are well developed, withpreliminary conclusions and recommendations,will also allow the Feasibility Study CommitteeInternational Foundation for Electoral Systems 45

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLE• Presentations by each vendor attending.• Question and answer session after eachvendor presentation.• Open viewing and testing of vendorequipment.• Individual closed meetings between theFeasibility Study Committee and vendors.In order to ensure transparency in theprocess the Feasibility Study Committee, orsponsoring institution for the feasibility study,should issue a press statement after thevendor demonstration. The statement shouldoutline what happened and any preliminaryconclusions from the discussion which canbe shared.The Feasibility Study Committee mightconsider using the vendor demonstration asan opportunity to test the usability of differentvendor solutions and the reaction of a sampleof the voting population to the technologies. 76Participants at the demonstration could bebroken down into groups of voters, onegroup for each electronic voting or countingtechnology being tested. Each groupwould vote using its designated electronicvoting or counting solution and fill in aquestionnaire about their experience withthe technology. The groups could rotate andtry other technologies, providing feedbackafterwards. 77 This process could provideinvaluable pre-pilot information on the usabilityof different options, as well as boost theconfidence and interest of stakeholders in theconsideration of these technologies.Should the vendor demonstration be used toconduct such testing, vendors would needto be informed well in advance so they couldprepare and adapt their electronic votingand counting solutions for this test, possiblytranslating user interfaces and instructions.The actual testing would also need tobe well organized to avoid a chaotic andunprofessional impression. Questionnaireswould need to be developed for those testingthe technologies.Stakeholder ConsultationAs identified earlier, it is essential thatstakeholders participate in the feasibilitystudy process so they can understand thework of the Feasibility Study Committee.Their participation also ensures they havethe opportunity to present their opinionsand concerns about the possible use ofelectronic voting and counting technologies.This inclusion and openness is more likelyto lead to acceptance of the resultingrecommendation by the Feasibility StudyCommittee and should ensure that thoserecommendations take into considerationa wide range of perspectives in the use ofelectronic voting and counting technologies.76 Although it would have to be understood that thesample of voters present at the demonstration wouldin no way be representative of the voting populationin general. Nevertheless it could provide interestingfeedback on the electronic voting and countingtechnology options being demonstrated.77 This process was used by the Philippine electionThis process was used by the Philippine electioncommission, COMELEC, at a vendor fair prior tointroducing electronic counting machines for its 2010elections.International Foundation for Electoral Systems 47

INTERNATIONAL DECISION PRINCIPLEELECTORAL STANDARDSAt a minimum, consultation should beconducted with political party and civil societyrepresentatives, especially domestic observerorganizations. However, this consultation couldalso be extended to key media representatives,political science institutes, governmentstakeholders and technology industry leaders.The purpose of including technology industryrepresentatives would be to encouragethese representatives to consider developingdomestically produced electronic voting andcounting solutions. These may be cheaperand more sustainable in the long term (it isrecognized that the development of suchcapacity is a long term project).It may be possible to combine thestakeholder consultation with the vendordemonstration, although it is preferableto hold them separately. Holding the twoevents separately will allow stakeholders,if invited to the vendor demonstration,the opportunity to carefully consider theinformation presented to them during thedemonstration, rather than requesting animmediate decision if the technologies theyare presented with are suitable.Even if a broad range of stakeholders wereinvited to the vendor demonstration it willstill be important to initiate the consultationwith presentations on the challenges andopportunities presented by electronic votingand counting technologies, the globalexperience of using these technologies, thework and preliminary findings of the FeasibilityStudy Committee and its working groups, anyrequirements that have been developed andpotentially a list of key questions that need tobe considered during the consultation.The Feasibility Study Committee may alsowant to consider the possibility of a muchmore broad ranging consultation than aface-to-face consultation mechanisms. TheCommittee might consider accepting writtensubmissions from any interested party asa way to gauge general voter attitudes andconcerns about the possible use of electronicvoting and counting technologies.According to the sequence provided in thisguide, the consultation with stakeholdersis conducted quite late in the process ofreaching a decision in principle. In principle,stakeholders should be included in theprocess as early as possible. The reasonfor suggesting that stakeholders not beincluded in the process earlier is based onthe assumption that the EMB is consideringthe issue of electronic voting and countingtechnologies for the first time. If this is thecase, then introducing stakeholders withlittle understanding of technology issuesinto the process too early will likely lead toan uninformed discussion about the likelybenefits and operation of any electronicvoting or counting solution. The FeasibilityStudy Committee will need to guidestakeholders in this debate. It can only dothis once informed and has had a chance toconsider the range of issues involved. Shouldstakeholders be much better informed aboutthe use of electronic voting and countingtechnologies, 78 then stakeholder consultationmay be considered earlier in this stage of thefeasibility study process.78 For example if EVM solutions had been used inprevious elections.48Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL DECISION IN STANDARDSPRINCIPLEDecision in PrincipleThe decision in principle will be a result ofall of the issues identified earlier — technicalfeasibility, beneficiality, financial feasibility andstakeholder acceptance. These issues willhave been explored fully through the stepsdiscussed above. These various factors willhave to be balanced against each other inorder to reach the decision in principle.A positive technical feasibility assessmenton using electronic voting or countingtechnologies in an electoral process will berequired for further steps. If electronic voting orcounting technologies are technically feasibleand supported by stakeholders then thedecision in principle may be that there shouldbe no further steps to implement if the benefitsto be achieved are not sufficiently greater thanthe disadvantages or the cost is too excessiveor does not justify the expected benefits.Even if the technologies are technicallyfeasible, provide significant benefits overthe existing system and are not excessivelyexpensive, the decision may still be taken tonot proceed if there is significant stakeholderconcern or resistance to the introduction ofthese technologies. While it is not impossibleto implement such technologies without thesupport of key stakeholders, to do so wouldbe a risky strategy potentially leading to awasted investment in electronic voting andcounting technologies.The Feasibility Study Committee willneed to assess other less tangible costsand benefits, such as public and politicalperception. The Committee may need toconsider both change management and riskmanagement strategies in order to addressissues identified during such an assessment.It should be noted that a fiscal cost benefitanalysis resulting in a favorable outcome foran electronic solution may not mean that thetechnology should be used. For example,the risk of politicians banning the use ofsuch technologies late in the game or a highlogistical risk of not being able to distributeelectronic voting or counting machines intime — indicate the potential “costs”, e.g.,delay of elections, are too high.Ultimately the decision in principle is avery difficult one to determine and a rangeof factors need to be considered by theFeasibility Study Committee. It should berecognized that to take an affirmative initialdecision in principle does not commit theEMB to anything at this stage. The nextstages in the feasibility study process areexperimental. Therefore, a decision toproceed to these next stages does notmean that a decision has been made to fullyimplement the technology.If some of the issues in reaching the decisionin principle indicate that electronic voting andcounting technologies should not be pursuedand instead relied on a number of assumptionswhose accuracy was not certain, then a pilotproject could be pursued to verify the validity ofthese assumptions. For example, uneducatedvoters would not understand how to use anelectronic voting machine.Whatever decision is reached at this stageof the feasibility study it will be important toensure that the reasoning behind the decisionInternational Foundation for Electoral Systems 49

DECISION IN PRINCIPLEis clearly elaborated by the Feasibility StudyCommittee, including any assumptions. Thisensures that even if the decision in principleis to not proceed with investigating the use ofelectronic voting and counting technologies,the work invested in the feasibility study canbe used in the future as a starting point forreconsideration if requirements, financialconsiderations or electronic voting andcounting products change.Should the Feasibility Study Committeedecide there is sufficient reason to continueits consideration of using electronic votingor counting technologies, then it will need torecommend that a pilot project be conductedand clearly define the mandate andparameters for this pilot. There are, however,a number of prerequisites that need to be inplace before the actual pilot can be initiated.50Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

Pilot PrerequisitesPilot testing for new voting and counting technologies may take place inmock polls or during scheduled elections.It is important to recognize there are certain issues that need to be addressed before anypilot project can be initiated. Other prerequisites are essential if the pilot is to be as effectiveas possible. These issues are fundamental to the way in which the pilot project is planned andconducted and should be established before this pilot process starts.Pilot Project MandateIt is essential that any pilot project conducted is provided a clear mandate. There are a numberof issues that will need to be defined to provide this clear mandate — the type of pilot projectto be conducted, pilot locations, technological solutions that should be piloted (single solutionor multiple solutions) and the issues that need to be explored in detail through the pilot.Type of PilotThe type of pilot can vary in a number of different ways and situations. Options in this regardare as follows:• Mock Pilot — Electronic voting and counting technologies solutions could be piloted inan entirely different electoral situation, a mock electoral situation. However, piloting thesetechnologies in a mock electoral situation is likely to skew the results of the pilot as itmay result in a very different profile of electorate taking part (only interested, educatedcitizens) than the normal voters who would need to use the electronic voting or countingtechnologies. This may mean that the results of the pilot are not a good representation ofInternational Foundation for Electoral Systems 51

INTERNATIONAL PILOT PREREQUISITESELECTORAL STANDARDShow the electorate in general would copeand respond to these technologies.• Parallel Pilot — Electronic voting andcounting technologies could be pilotedalongside an existing voting processsuch that all voters cast their ballots asnormal using the existing system. Theywould also have the opportunity to casta mock ballot, or maybe the same ballot,afterwards, using the technology. Theresults of the parallel pilot would have noforce with respect to the result. Again,the kind of voters who might exercisetheir right to participate in this parallelpilot may not be representatives of theaverage electorate, with uneducatedvoters not taking the opportunity asoften as more educated ones. However,this approach is more likely to survey abroader cross section of voters than themock pilot.• Optional Pilot — Electronic voting andcounting technologies could be pilotedalongside the existing voting process,with voters having the option to eitheruse the existing system or the electronicvoting or counting system. The resultsfor the polling station would then becalculated by combining the results fromballots cast using the existing systemand the electronic voting or countingsystem. Votes cast using the technologywould have equal validity as those castusing the existing system. Again thisoptional pilot may result in only certainmembers of the electorate using theelectronic voting or counting technology.• Compulsory Pilot — This type of pilotexclusively uses electronic voting andcounting technologies for selectedmembers of the electorate. These voterswould have to cast their ballots usingthe technology and these votes wouldprovide part of the overall result.Clearly the best option for obtaining adefinitive assessment of how the generalelectorate responds to using electronicvoting and counting technologies is wherea section of the electorate is required to usethe technology being piloted and is not ableto opt out. However, this is also risky. If theelectronic voting or counting solution beingpiloted is defective in some way or is seento favor some of the electorate over others,then its compulsory use could be challengedin the courts at a later date. A successfulchallenge could call into question the validityof the election result in which the pilot wasconducted and possibly require a repeatelection to remedy the situation. 79Pilot LocationsThe mandate will need to define the scale ofthe pilot to be conducted, in terms of numberof locations that it will be held in, and someparameters as to where these locationsmight be.Where an in-person electronic voting orcounting solution is being piloted, (e.g., asolution in an in-person polling location as79 See the example of Finland, where a problemin the confirmation of the vote using electronicvoting machines during a 2008 pilot project led toa challenge in the court and new elections beingordered in the three pilot project municipalities – seeCouncil of Europe (2010), p. 20.52Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT PREREQUISITESSTANDARDSopposed to internet voting replacing votingby mail), then the identification of the voterswho will participate, or have the opportunity toparticipate, in the pilot will be clear. All of thevoters registered to vote at that location, or asubset of this group, will participate in the pilot.It is advisable that electronic voting andcounting technologies be piloted inmultiple locations, so that a cross sectionof the electorate can test the use of theselected technologies. This will requirethat consideration be given to the differentkinds of voters that should be provided theopportunity to test the use of the electronicvoting or counting technologies. For example,only testing electronic voting or countingtechnologies in urban locations would notbe advisable as rural voters may have a verydifferent reaction to using these technologies.It may well also be that there are a range ofenvironmental factors in which electronicvoting and counting technologies need tobe tested, and therefore pilot locations willneed to be selected accordingly. Initial pilotsmay also be chosen for constituencies/areaswhich are not contentious politically so as toavoid politically charged scenarios and allowtrust to build in the pilot technologies. If thesituation permits, a pilot could be conductedfirst in a single location to primarily test theEMB’s ability to cope with the new process,procedures, training, voter education andlogistical requirements. Subsequent pilotscould be conducted at a number of locationsrepresenting a broader variety of the electorate.Piloting remote electronic voting solutions,such as internet voting, may require a differentapproach to selecting pilot participants.The selection of participants for a remoteelectronic voting pilot may be limited by voteridentification mechanisms that the remotevoting system would utilize. Or the remotevoting solution may be targeted at a specificsection of the electorate, such as votersabroad, indicating that this entire groupshould take part in the pilot project.Solutions Being PilotedThe decision in principle may indicate thatone electronic voting or counting solution bestmeets the needs of the electoral process.This does not mean that it has to be the onlysolution piloted. Likewise, if a specific type oftechnology is being piloted (e.g., electroniccounting of scanned ballots), then thisdoes not mean that several other countingsolutions cannot be tested as part of the pilot.The mandate may indicate which specifictechnology is to be piloted and if a rangeof solutions or a single solution is to bepiloted. It is recommended that morethan one electronic voting and countingsolution be piloted. This is important ifthis is the first time these technologies arebeing investigated; allowing for greaterunderstanding of the various systems.Where the solutions to be piloted have notbeen made clear in the mandate, this needsto be determined at the early stage of thepilot project management process.LegislationThe process of taking the decision inprinciple should have identified if the existingelectoral legal framework permits the use ofInternational Foundation for Electoral Systems 53

INTERNATIONAL PILOT PREREQUISITESELECTORAL STANDARDSelectronic voting and counting technologies,or whether changes are required to allowtheir use. If existing legislation does not allowthe use of electronic voting and countingtechnologies then the types of pilot identifiedabove (Optional or Compulsory) will not bepossible until legislation is changed to allowthese technologies to be used.“It is recommended thatmore than one electronicvoting and counting solutionbe piloted.”Where legislative changes are required, theycan be temporary in nature for a specificelection at which the pilot will take place soeither the existing system of balloting can beused or an electronic system can be used.The latter approach provides maximumflexibility for the pilot process and meansnew legislation does not need to be passedfor each election in which a pilot takes place(it is entirely possible that electronic votingand counting technologies could be pilotedover several elections). However, changingelectoral legislation so that either system couldbe used by the EMB might be seen as aninvitation to use electronic voting and countingtechnologies at the discretion of the EMB.In addition to legislative changes requiredto allow the use of electronic technologies,it is almost certain that electoral regulationswill need to be changed. In most electoraljurisdictions these regulations are passedby the EMB, so changing them is lessproblematic than changing electorallegislation. It is still essential that theregulations be amended to facilitate the useof electronic voting or counting technologies.Electronic Voting andCounting TechnologySpecificationThe steps conducted during the decisionin principle process will help the FeasibilityStudy Committee, and the EMB, to ensurethat any electronic voting and countingtechnology pilot process is driven by theactual needs of the electoral process. Therequirements, previously defined, will becentral to drafting a comprehensive requestfor proposal for the electronic voting orcounting technology procurement process.The request for proposal will need toidentify the technical specifications whichthe solution must comply with for it to beconsidered and also request information onother product and support related issuesrelevant to the bid selection process.The technical specification will need toprovide the following parameters for vendorsto comply with:• Type of electronic voting or countingsolution for which quotes are beingrequested (e.g., electronic voting,electronic counting, remote votingsolutions, etc.).• Scale of the pilot, including number oflocations, number of voting or countingmachines required, scope of any remotevoting pilot and number of registeredvoters the pilot will need to accommodate.54Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT PREREQUISITESSTANDARDS• Details of any paper audit trail, or othermechanisms for auditing or verifying theaccuracy of the result 80 , required by theelectronic voting or counting solution.• Quantity of electronic voting or countinghardware and software that will likely beprocured, and whether this is likely to besplit between one or more suppliers.• Power requirements related to usingelectronic voting or counting hardwarein locations which do not have reliablemains power sources.• Electoral systems that need to beaccommodated by the electronic votingor counting technology (e.g., first past thepost, list based proportional representation,preference voting, bloc voting, etc.).• Requirements for coping with multiplelanguages, the scripts that will needto be accommodated and any specificrequirements to ensure electronicvoting or counting solutions are easy touse for voters. 81• Details of any environmental conditionsthe electronic voting or countinghardware would have to be able to dealwith, including extremes of heat, cold,humidity and dust.• Security requirements for the electronicvoting or counting technology. 8280 The Council of Europe (2004) recommendationon e-voting requires that the correctness of theresult produced by an e-voting system should beverifiable and that the system should be auditable –recommendations 26, 59 and 100-110.81 The Council of Europe (2004) recommendationon e-voting requires that the voter interface of anelectronic voting system be understandable andeasily usable – recommendation 1.82 The Council of Europe (2004) recommendation one-voting requires that the reliability and security ofan e-voting system be ensured and that all possiblesteps be made to mitigate against fraud using thesystem – recommendation 28 and 29.• Services that will be required from thevendor during the conduct of the pilotproject in addition to delivery of theelectronic voting or counting solution(e.g., project management services,configuration, training, and service supportduring the voting period in the pilot).• Anticipated delivery times for all servicesand goods to be provided once acontract has been awarded.• Project management arrangementsthat would be put in place by thevendor to coordinate pilot projectimplementation issues, clearly identifyingthe responsibilities that would be coveredby these arrangements.Additional information will also be required forthe selection process such as information notdirectly covered by the requirements for change(previously identified). This information mayrelate to basic functionality of the electronicvoting or counting system, functionality that allsystems will have, but will likely be implementeddifferently on each machine.Therefore, the request for proposals shouldask for information from the vendors on thefollowing:• Intellectual property rights the vendorwill expect to exercise over the hardwareand software provided as part of the bid.International Foundation for Electoral Systems 55

INTERNATIONAL PILOT PREREQUISITESELECTORAL STANDARDS• Will the election management systemthat accompanies the electronic votingor counting solution allow configurationof the machines for specific elections,lists of candidates and electoral systemsbeing used?• Can an electronic voting machinesolution include an electronic voterlist and integrated biometric voteridentification system? If so, what arethe mechanisms that ensure the votercannot be linked to the vote?• Ways in which the electronic voting orcounting hardware would cope with asudden loss of power, ensuring that datais properly secured.• Means by which electronic voting orcounting solutions are activated for useby voters to ensure that voters can onlycast the correct number of votes. 84• Maximum capacity of the electronicvoting or counting solution in termsof number of electoral races andnumber of candidates that can beaccommodated (in the case of screenbased electronic voting machines itmay be relevant to request the totalnumber of candidates that could beaccommodated on a single page of thescreen with a specified font size).• Means of verifying that loaded softwareis the same as that tested and approvedby the EMB.• Means by which the electronic ballot boxcan be verified as being empty at thestart of polling.• Means by which it can be demonstratedbefore the start of polling to observersand party/candidate agents thatthe machine counts votes/ballotsaccurately. 8383 The Council of Europe (2004) recommendation one-voting requires that observers be able to be presentto observe and comment on the e-elections, includingthe establishing of the results – recommendation 23.• Whether the electronic votingtechnology can accommodate thedisplay of party and candidate symbolsand photographs.• Mechanisms for review and confirmationof the ballot choices made by the voter. 85• Whether the solution has the possibilityfor accepting blank ballots, or a ‘none ofthe above’ option on the ballot. 86• Ways in which the electronic votingsolution can provide access to peoplewith disabilities. 8784 The Council of Europe (2004) recommendation one-voting requires that voters shall be prevented fromcasting more than one electronic ballot or from castingballots on multiple channels (paper and electronicallyfor example) – recommendations 5 and 6.85 The Council of Europe (2004) recommendationon e-voting requires that voters be prevented fromcasting their ballot without reflection and to alter orterminate their vote before completion of the e-votingprocess – recommendation 10 and 11.86 The Council of Europe (2004) recommendation one-voting requires that the casting of a blank voteshould be possible – recommendation 13.87 The Council of Europe (2004) recommendation one-voting states that e-voting systems should bedesigned to maximize the opportunities for access forpeople with disabilities – recommendation number 3.56Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT PREREQUISITESSTANDARDS• Mechanisms for ensuring voters havecompleted the casting of their ballot whenthey leave an electronic voting solution. 88• Whether an electronic voting machine(may not be relevant for electroniccounting machines) has the possibility forlimiting the speed with which ballots arecast in order to mitigate electronic ballotbox ‘stuffing.’• Means by which results can be transferred/transmitted from the machine for tabulationof results and the mechanisms by whichthese results can be verified as from alegitimate source on receipt. 91• What kind of software is used totabulate and publish the results ofelections using the electronic voting orcounting solution?• If a printer is used in an electronic votingmachine for a paper audit trail or to printthe ballot, what type of printer is used(thermal, laser, inkjet, etc.) and what arethe environmental limitations on its use?• Means by which the electronic votingor counting solution is designed to dealwith well established security challengespresented by the use of electronictechnologies.• For paper audit trails on an electronicvoting machine, how will the paper bechanged part way through polling?• Means by which the secrecy of anelectronic vote is maintained by thesystem. 89• Means by which results can begenerated by the electronic voting orcounting solution at the end of polling. 9088 The Council of Europe (2004) recommendation one-voting requires that it is clearly indicated to the voterthe voting process has been completed.89 This is a general international electoral standardrepeated in the Council of Europe (2004)recommendation on e-voting, but also expandedon to require that in an electronic ballot box it is notpossible to reconstruct the link between the voterand the vote cast, and furthermore that any remotee-voting system should not allow a voter to prove thecontent of their vote – recommendations 16, 17, 51and 52.90 The Council of Europe (2004) recommendation one-voting requires that the counting process shallaccurately count the votes, and that the e-votingsystem maintain the availability and integrity ofthe electronic ballot box and counting process –recommendation 98 and 99.• Mechanisms that are in place to ensurethat if electronic voting or countingsystem has been tampered with, it canbe detected. 92• Life expectancy of the electronic votingor counting solution and the period ofvendor guarantee for the hardware.• Maintenance and storage requirementsto ensure that the electronic votingor counting solution is kept in goodworking order.It will be necessary to define the technicalrequirements for the electronic voting or91 The Council of Europe (2004) recommendationon e-voting states that the integrity of datacommunicated during the voting stage be maintained,and that data-origin authentication be carried out –recommendation 97.92 The Council of Europe (2004) recommendation one-voting requires that sufficient means be providedto ensure that the systems used to cast the voteare protected against modifying influences –recommendation 92.International Foundation for Electoral Systems 57

PILOT PREREQUISITEScounting solution prior to the commencementof the pilot as many of these components willbe fundamental to the way in which the pilotis conducted.In addition to the information sought in thisrequest for proposal, vendors who submitproposals should be required to commit toimplementing their solutions during the pilotin accordance with good practice for theconduct of elections.Pilot Project FundingThe conduct of a pilot project will entail anumber of costs, the least of which may bethe procurement of any electronic voting orcounting equipment itself. A budget will needto be developed for the conduct of the pilotproject. The budget will depend a lot on thescale of the pilot being recommended, andcan draw heavily on the costs identified bythe working group looking at the financialaspects of using these technologies.It may be that the budget for the pilot projectwill be drafted at the same time the decisionin principle to proceed with a pilot is taken.It should almost go without saying that theprocess of implementing a pilot projectcannot start before the budget required toconduct the pilot has been secured.58Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

Pilot ProjectPilot testing voting and counting technology is essential to confirming thechallenges and benefits of a new system.Piloting electronic voting and counting technologies is a way of testing many of theassumptions and conclusions reached during the process of reaching a decisionin principle. This includes a practical assessment of actual benefits and disadvantagesin using the piloted electronic technologies, the actual costs involved in implementingthese technologies and the suitability of the list of requirements developed for electronictechnologies. The pilot will also allow the Feasibility Study Committee to assess issues whichcould only be guessed at during the decision in principle stage of the process, including theability of voters to properly use the new technology.A good pilot will need to take into consideration the following issues.Managing the Pilot ProjectImplementation of an electronic voting and counting technology pilot project is an incrediblycomplex task. It requires a good project management structure to ensure that it is plannedeffectively and that timelines and objectives are continuously monitored and amended asrequired. The implementation of the pilot will require a lot of components of the EMB towork effectively together, calling for significant commitment from the EMB to deliver on thevarious aspects of the project. This guide will not provide a specific framework for projectmanagement of the pilot since there are sufficient guides available.International Foundation for Electoral Systems 59

INTERNATIONAL PILOT PROJECTELECTORAL STANDARDSAt a minimum, it is suggested that thefollowing project management structures beestablished to implement an electronic votingand counting technology pilot project:• Pilot Project Committee — This shouldinclude at least some members of theFeasibility Study Committee (assumedto be the case in this guide) and seniormembers of the EMB. It will also beimportant to include representativesfrom all of the components of the EMBwhich will need to have an active rolein implementing the pilot project. Thismay include representatives from fieldoperations, training, logistics, votereducation, legal and IT sections.• Pilot Project Manager — Day-to-daymanagement of a complex projectcannot be done by a Committee.Therefore, proper management of thepilot project will require that a competentstaff member be identified as the focalpoint for pilot project activities. Thisproject manager would need to be fulltime on the pilot project, empowered withauthority to take decisions while reportingback to the Pilot Project Committee ondevelopments and compliance with theimplementation plan.The issue of contact with vendors prior tothe issue of a contract for the supply ofelectronic technologies is worth mentioningagain. Contacts with vendors will be requiredas the procurement process is under way.The Pilot Project Committee will need toensure it deals fairly and equally with vendorsand protects itself from any allegations offavoritism and inappropriate conduct withrespect to vendors. A single point of contactshould be established, with communicationsapproved by the Pilot Project Committee.Risk Management PlanThe Council of Europe recommends thata risk management plan should alwaysbe developed for an electronic voting andcounting technology project. 93 This is goodpractice for any project, but especially sofor electoral projects where timely deliveryof voting services are so critical. The riskmanagement plan should cover the followingpotential difficulties if or when they occur: 94• Equipment is late or missing.• Equipment breaks down.• Internet connection fails.• Access to the voting website or the voterregister fails due to firewalls or othersecurity measures.• There is a software error.• Polling station personnel do not arrive.• There is a power failure.• There is a natural disaster or otheremergency.• Challenges in securing the necessaryrecruitment of skilled human resources.93 Council of Europe (2010), p. 31.94 Mainly drawn from Council of Europe (2004) p. 31,but with a few additions.60Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT STANDARDS PROJECT• Possible legal challenges to the useof electronic voting and countingtechnologies.This risk management plan should beestablished at the beginning of the pilotproject process and reviewed periodicallythroughout the process. The same riskmanagement plan — as adjusted basedon lessons learned during the pilot —should be used during any future full scaleimplementation of the electronic voting andcounting technologies.Procuring Electronic Votingand Counting TechnologiesThe process of procuring electronic votingand counting technologies can take sometime and needs to be conducted in an openand transparent manner. The EMB needsto ensure it is in control of this procurementprocess in terms of defining the requirementsfor the technologies to be piloted. The processmust not be vendor driven, with vendorstelling the EMB what it is that they require.Good procurement practice would indicateadhering to the following three procedures:• Time — Sufficient time will need to beprovided to vendors to properly respondto the many facets of the request forproposals. A reasonable timeframe forsuch a request for proposals would be inthe region of four to six weeks. Vendorsshould be allowed to seek clarificationson aspects of the request for proposalsat a predefined date part way throughthe procurement process. The questionsraised and answers provided should bemade available to all through the EMB or asponsoring institution’s website, includingvendors who submitted questions.• Open and Impartial ProcurementProcess — The procurement processitself should be open and impartial.Request for proposals should be widelypublished through the media and onthe sponsoring institution’s website;decisions should be taken according topre-established criteria. In addition, itmay also be decided to send the requestfor proposals directly to well-establishedelectronic voting and countingtechnology vendors, especially thosewith whom a relationship has been builtduring the decision in principle stage ofthe feasibility study.• Evaluation Criteria — The request forproposals should indicate to the bidderswhich evaluation criteria will be applied inselecting vendor(s) to supply products forthe pilot project. Evaluation criteria mightinclude compliance of the proposal withtechnical specifications, experience ofthe vendor in delivering similar solutions,quality and experience of the projectmanagement team offered by the vendor,access provided by the vendor to thesource code and the price.Each criterion needs to be individuallyscored (e.g., on a scale of 1–10, with 1indicating the lowest score and 10 thehighest). Each evaluation criterion willnot be of equal importance; therefore,International Foundation for Electoral Systems 61

INTERNATIONAL PILOT PROJECTELECTORAL STANDARDSthe criteria will need to be weightedto provide an overall scoring for eachproposal received. An example ofsuch a weighting could be technicalspecifications (50 percent), vendorexperience (15 percent), projectmanagement team (15 percent) andprice (20 percent). A sample of how toimplement such a weighted ranking forproposals is included in Annex 4.Other typical decisions to be determinedin establishing such a selection procedureare whether to make the weightingsknown to the bidders and whether there isa minimum requirement for consideration,either for each category or in total. Insome cases financial proposals areonly accepted for bids achieving a setminimum score in the technical evaluation.It is clear that the specification and resultingproposals will be complex and detaileddocuments. A Proposal Review Committee,possible the entire Pilot Project Committee(depending on the size of this Committee),should review the proposals received andagree on the ranking against differentevaluation criteria. On the basis of this, arecommendation will be made on whichelectronic voting or counting solution, orsolutions, will be procured for the pilot project.Testing and CertificationOnce delivered, it is essential that an EMBensure that an electronic voting or countingsystem not only meet the specificationsdeveloped for the system, but also meet therequirements of the electoral environment.The CoE’s E-Voting Handbook identifies sixtypes of testing that the EMB should conduct.Quoting from the CoE Handbook, these are:• Acceptance testing — This is a methodof testing software that tests thefunctionality of an application performedon a system (for example software,batches of manufactured mechanicalparts, or batches of chemical products)prior to its delivery.• Performance testing — This test is usedto determine the speed or effectivenessof a computer, network, softwareprogramme or device. This processcan involve quantitative tests done ina laboratory, such as measuring theresponse time or the number of MIPS(millions of instructions per second)at which system functions. Qualitativeattributes such as reliability, scalabilityand interoperability may also beevaluated. Performance testing is oftendone in conjunction with stress testing.• Stress testing — This is a form of testingused to determine the stability of agiven system or entity. It involves testingbeyond normal operational capacity, oftento breaking point, in order to observe theresults. Stress testing may have a morespecific meaning in certain industries,such as fatigue testing for materials.• Security testing — This is a process todetermine that an information systemprotects data and maintains functionalityas intended. The six basic securityconcepts that need to be covered by62Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT STANDARDS PROJECTsecurity testing are: confidentiality,integrity, authentication, authorisation,availability and non-repudiation.• Usability testing — This is a techniqueused to evaluate a product by testingit on users. This can be seen as anirreplaceable usability practice, since itgives direct input on how real users usethe system.• Review of the source code — This is asystematic examination of the computersource code intended to find andrectify mistakes overlooked in the initialdevelopment phase, improving both theoverall quality of the software and thedevelopers’ skills. 95Conducting all these tests takes time and itis important that time for full testing is madeavailable in the project timeline.In addition to comprehensive testing ofelectronic voting and counting technologiesprior to use, it is increasingly seen as goodpractice to have these systems certifiedprior to use. 96 The purpose of certification issimilar to testing in that it determines whetherthe electronic voting or counting technologyoperates effectively. The difference is that itis conducted by an authority independent ofthe EMB, political parties, the governmentand suppliers. Ideally the certification process95 CoE (2010), p.34-35.96 The Council of Europe (2004) recommendation one-voting requires that before any e-voting system isintroduced, it be certified by an independent bodyto verify that it is working correctly and meets allnecessary security measures – recommendations 25and 111.will be conducted in an open and transparentmanner, and is intended to build confidencein the operation of the electronic technology.Time is again an issue, and the process ofcertification may take between 6-12 months,depending on how many issues are foundwhich require fixing. 97While a number of institutions could playa role as certifying institutions, such asuniversity departments of informationtechnology or technology institutes, theprocess of certification will need to be welldefined. In some countries the certifyinginstitutions themselves have to be preauthorizedand must meet a series ofstandards for the work they will conductcertifying electronic voting and countingtechnologies. Clear guidance will need tobe developed for certifying institutions onhow they are to conduct the certificationprocess, the records they should makeof their findings, the consequences of aproduct failing to comply in some way, themechanisms for a vendor to resubmit afterfailing certification and the openness of thecertification process and certification reports.Polling and CountingProceduresMany aspects of electronic technologies willlikely be different from the existing systemof balloting, especially if the existing systemis a paper balloting system. The proceduresfor storage of the electronic voting orcounting machines, pre-polling preparations,transportation, security, placement in the97 CoE (2010), p.33-34.International Foundation for Electoral Systems 63

INTERNATIONAL PILOT PROJECTELECTORAL STANDARDSpolling station, demonstrating an emptyballot box, initiating polling, activation of theelectronic voting or counting machines for thevoter and reporting of results will be different.These changes in procedure will need tobe carefully considered by a competentand experienced group of electionmanagement officials, in consultation withother stakeholders. Full procedures for eachamended part of the process will need to bedeveloped and tested before they are readyto be used in a pilot.The type of pilot being conducted will notchange the need for carefully considered andcomprehensive procedures for the use ofelectronic technologies during the pilot process.While the development of new procedures canbegin prior to procurement of the technology,finalization of the procedures can only be madeonce the exact technology is selected.Voter EducationEducating voters on the use of newelectronic voting or counting technologiesis essential, 98 and must start before they areconfronted with the new system on ElectionDay. A change in balloting system, especiallyif moving from paper balloting to an electronicvoting solution, will be confusing for voters.This confusion, and problems in usingelectronic voting or counting technologies,can be mitigated to a large extent by effectivevoter education in advance of the pilot98 The Council of Europe (2004) recommendation one-voting requires that information on the functioningof an e-voting system be made publicly available andthat voters understand and have confidence in thesystem – recommendations 20 and 21.project. There are a number of key messagesthat need to be communicated to voters inadvance of a pilot project:• Existence of a Pilot — Voters need tounderstand electronic voting or countingtechnologies are going to be used, thelocations they will be used in and whythe technologies are being tested.• Type of Pilot — Outlined previously, thereare a number of different types of pilotprojects that can be implemented —mock, parallel, optional and compulsory. Itshould be clearly communicated to voterswhich of these will be implemented.• Follow Up — Voters should be madeaware if a sample of those using theelectronic voting or counting systemduring the pilot will be surveyed bythe EMB to assess their experiencesof using the technology immediatelyafter completion. This way, they arenot surprised or suspicious whenapproached and asked questions whenleaving the polling location.“Educating voters on theuse of new electronic votingor counting technologiesis essential, and must startbefore they are confrontedwith the new system onElection Day.”The task of voter education will need tobe conducted in a targeted manner as thepilot will only be in a limited geographic64Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT STANDARDS PROJECTarea. Mass means of communication areless suitable as they would communicatemessages about the pilot to many whowould not be involved and therefore likelylead to confusion about where the pilotwill be conducted. This means local levelmechanisms for voter education should beemployed for the project.The CoE makes an additionalrecommendation with respect to votereducation before an electronic votingor counting solution is implemented. Itrecommends that voters be provided withthe opportunity to practice any new methodof electronic voting before and separatelyfrom the actual casting of the vote. This willhelp minimize voter anxiety about the newmeans of voting and promote understandingand confidence in the new system. 99 Thefeasibility of doing this will depend on thescale of the pilot project, but certainly votersand other stakeholders should be providedthe opportunity to sample using the systembefore Election Day. Such ‘testing’ of thesystem by voters could be done at the pollingstation prior to Election Day — making clearthat this is not the actual election — or bytaking the electronic machines to communitymeetings in advance of the pilot.TrainingJust as the education of voters in the useof piloted electronic voting and countingtechnologies is essential to the success ofthe pilot, so is proper training of staff who willuse the technologies. As already discussed,99 Council of Europe (2004) recommendation one-voting, recommendation 22.the procedures for many, if not most, aspectsof polling and counting may be changed bythe introduction of these technologies. Notonly must new procedures be developed, buttraining on these new procedures needs tobe effectively delivered.This training will be required not only bypolling staff, but also the staff required toprepare the electronic voting or countinghardware at centralized facilities and staff whoreceive the results provided by the electronicvoting or counting technology. As initial pilotsare generally small in scale, delivering trainingto the staff involved should not be too difficultfrom a logistics perspective.From a technical and procedural perspective,the preparation of training for the pilotneeds to go through all the steps of policy,procedure and material development that anyother aspect of polling operations would needto. Procedures need to be drafted and testedand training materials for these proceduresdeveloped. These procedures need to coverthe configuration of the hardware, setup ofany machines in the polling station, conductof polling, close of polls, production of results,transfer of results for tabulation and receipt ofresults for tabulation.The training provided on these newprocedures will need to be comprehensivesince these components will be significantlydifferent with electronic voting or countingsolutions when compared to paperballoting. While it is essential that electionadministration staff be provided this training,it would also be beneficial if similar briefingswere to be provided to candidates, partyInternational Foundation for Electoral Systems 65

INTERNATIONAL PILOT PROJECTELECTORAL STANDARDSagents and observers in pilot areas so theyalso understand the new procedure.Stakeholder OutreachGetting the support of key stakeholders willbe important to the perceived and actualsuccess of any pilot for electronic voting andcounting technologies. Providing access tothe technology prior to elections, as indicatedabove, will be one way of reaching out to keystakeholders. However, additional efforts toinform stakeholders should also be pursued.Local candidates, party representatives,domestic observers, media and communityrepresentatives should be briefed by theEMB on the pilot project at the beginningof the planning process. They will need tobe informed about the technology beingpiloted, the reasons why it is being pilotedand the benefits that it is expected to bringto the process.If stakeholders can be won over to the pilotprocess, they can be strong supporters ofthe process, acting as a channel for key votereducation information and providing vitalmechanisms for feedback on the success, orotherwise, of the pilot project.the pilot project, locations in which the pilotwill take place and the benefits that areanticipated from using electronic technology.It will be especially important to inform thosewho will be voting in pilot project locations.In line with this policy of openness the PilotProject Committee may also consider havingsome or all of its meetings open to the mediaand those interested. At a minimum it shouldissue press releases at key points in the pilotproject process.Election Day SupportThe piloting of an electronic voting orcounting system will likely involve manysignificant changes in the process ofadministering elections. Regardless of howgood the training and documentation thatis provided to electoral officials, there willinevitably be some problems in applyingthe procedures and training when electoralofficials come to use electronic voting orcounting systems on Election Day. Goodprocedures, training and documentationcan help mitigate these problems to a largeextent, but it always makes sense whenmaking significant changes to a system tomake special provisions for dealing withquestions concerning the new procedures.PublicityA pilot project on the use of electronic votingand counting technologies is by nature avery public process. The earlier that the PilotProject Committee can start to publicizethat it will be conducting a pilot, the better.It will be important to communicate to allstakeholders the rationale for conductingA dedicated, centralized help desk isa good way of dealing with the manyquestions likely to be raised whenimplementing the kinds of changes to votingprocedures that occur with the introductionof an electronic voting or counting system.The help desk should be available fromat least a few days before the conductof elections to deal with questions that66Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT STANDARDS PROJECTpolling officials may have as they are issuedelectronic voting or counting equipment.Having a uniform mechanism for dealingwith such questions will ensure that ifpersistent problems are being encounteredthe problem/issue will become apparent asearly as possible, allowing the greatest timeto develop and implement a technical orprocedural fix. The help desk also ensuresthat uniform responses and guidance aregiven to electoral officials if the same questionor problem arises in different locations.All calls and issues raised through the helpdesk should be logged, and this log of issuesand questions will provide important informationwhen the success of the pilot project isevaluated at the end of the pilot process.In addition, having qualified trainers andtechnical staff in the field on Election Day todeal with issues on site may be advisableif resources allow. They can provide an onsitehuman interface with the help desk.However, solutions offered on-site shouldbe in line with what is provided by the helpdesk, and any issues encountered arereported to the help desk.The help desk operators must be thoroughlytrained in all aspects of the electronic votingor counting system, they must have adetailed helpdesk manual available and ashared log of issues raised and solved. Theymust have a set method for dealing withissues not covered in manuals and training,which could include a direct hotline to oneor more senior election officials authorized tomake decisions as required.Observation of the PilotProjectThe same rights to observe the electoralprocess should be applicable to any electronicvoting or counting technology pilot project. 100The EMB may have to take additionalmeasures to facilitate and encourage thisobservation for a number of reasons.First, the conduct of elections using anelectronic voting or counting technologywill be very different and will require specialtraining for observers, media and politicalparty and candidate agents that wish toobserve the pilot. This training will be neededto ensure that these groups understandhow the new system works, but also thatthey understand how it is that they can andshould observe the conduct of electronicvoting or counting technologies.Furthermore, as observers, media and politicalrepresentatives are key stakeholders in theprocess. Their trust in the system being pilotedwill be essential and, therefore, they shouldbe actively encouraged to observe. This willbuild their understanding of the system beingpiloted and allow them to provide feedbackto the Pilot Project Committee during the pilotproject evaluation stage.Mandatory AuditAs discussed earlier, the ability to verifythe operation and audit the results of anelectronic voting or counting system is an100 The Council of Europe (2004) recommendation one-voting requires that observers shall be able to bepresent to observe and comment on e-elections,including the establishment of the results –recommendations 23 and 56.International Foundation for Electoral Systems 67

INTERNATIONAL PILOT PROJECTELECTORAL STANDARDSemerging standard with respect to electronicvoting and counting technologies. 101 Theway in which this auditability is provided forwill vary depending on the type of electronicvoting or counting solution in question (e.g., itwill be different for electronic voting systems,electronic counting systems and especiallyfor remote electronic voting systems). Themost common way in which auditability isachieved for electronic voting systems 102 isthrough the use of a voter verified paper audittrail, which can be manually counted as acheck against the electronic result generatedby the electronic voting machine.Regardless, an audit mechanism is a wayof checking that the technologies workedproperly, by comparing the electronic andauditable versions of the results. In additionto checking the operation of the system, thisalso helps build confidence in the system,more so if the audit is done under the fullobservation of stakeholders in the process.For the pilot, therefore, the conduct of auditsof the results generated by the electronicvoting or counting system should bemandatory. 103 The (paper) audit trail should bemanually counted and the results compared101 Council of Europe (2004) recommendation one-voting, recommendations 26, 59 and 100-110.102 Auditability is mainly an issue for electronic votingsystems, as electronic counting systems normallyuse a paper ballot completed by the voter, whichnaturally provides a paper audit mechanism.103 This assertion is supported by the Council of Europe(2010) in its E-voting Handbook where it recommendsthat a paper audit trail should be combined with amandatory count of paper votes in a small, butstatistically meaningful number of randomly selectedpolling stations, p. 12. This guide is written withrespect to actually implementing e-voting systems,but it is equally relevant for the conduct of any pilotsusing e-voting systems.to the electronic results generated. Ideally thisaudit will take place in every location wherethe technology was piloted. This may notbe possible for a larger pilot project. If only asample of pilot locations are being auditedit will be important to randomly select thissample and only make the selection knownafter the close of polling and counting.The audit process should be conducted assoon as possible after the pilot. An audit rightafter the close of voting and counting avoidsthe possibility or perception of tampering ormanipulation before the audit takes place. Ifan immediate audit is not possible then thesample to be audited should be sealed in away which would be tamper resistant until theaudit can take place. The audit should be fullyobservable by election observers, the mediaand political party and candidate agents.The results of the audit process will needto be interpreted differently dependingon the kind of technology being piloted.With electronic voting technologies thereshould be no differences at all between theresult generated from the audit trail and theelectronically generated result. If a differenceis found then it will be prudent to conduct arecount of the audit trail to make sure thatthe manual process has not generated amistake. Should a difference between themanual count of the audit trail and electroniccount of votes still persist, even if only byone vote, this will be seen as an indication ofsome flaw in the operation of the electronicvoting machine or the audit trail. Even a smalldeviation would be a critical concern. Withoutunderstanding why the difference had beenpossible it also could not be known if this68Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT STANDARDS PROJECTflaw could lead to much larger deviationsbetween the electronic result and audit trailresult on future occasions. Additionally,even one vote incorrectly recorded is aserious violation of electoral standards and adisenfranchisement of the voter(s) affected.assessment needs to be conducted from theperspective of every key stakeholder in theprocess. Perceptions of these stakeholdersabout the use of the electronic voting orcounting technologies will be critical to anyfuture adoption of the technology.With electronic counting technologies,the interpretation of differences betweenthe manual recount of the audit trail andelectronically generated results are moredifficult. Paper ballots are marked in differentways by different voters, and sometimesthese voter marks are interpreted differentlyby electoral officials. The advantage ofelectronic voting technologies is that theyinterpret ballot marking in a consistentmanner, according to the instructionsprovided to them. A difference in vote totalsthrough a manual count of the ballots isprobably due to the counting machinereading voter marks in a different way thanthe election official. It may be the electionofficial has made a mistake, or it could bethat the difference represents an error in theballot counting rules provided to the countingmachine. This requires an amendment to thecounting machine software. Depending onthe severity of any error in the ballot countingrules provided to the counting machine,this may have implications, even seriousimplications, for the results generated bycounting machines across the election.Pilot Project EvaluationA comprehensive post-pilot assessment ofthe pilot project is essential. It would not beenough to conclude that polling seemedto go smoothly, if it did. The post-pilotThis pilot project evaluation needs tocollect opinions, at a minimum, from thefollowing stakeholders:• Voters who used the electronic voting orcounting technology.• Voters who did not use the electronicvoting or counting technology.• EMB staff involved in preparing theelectronic voting or counting technologyfor use.• Polling staff using the electronic voting orcounting technology.• Election management staff involved inthe receipt and tabulation of results.• Observers (domestic and international).• Candidates, and candidate and partyagents.• Representatives of other keystakeholders with a specific interest (e.g.,people with disabilities if special votingmechanisms are being implemented forsuch voters).The results of the pilot project will needto be assessed using many differentInternational Foundation for Electoral Systems 69

INTERNATIONAL PILOT PROJECTELECTORAL STANDARDSThe pilot project will offer insight into the ease of setting up and using newtechnology in a variety of conditions, including polling in remote locations.methods, from statistical data collectedabout the use of the electronic voting orcounting technology to qualitative analysisof the process from the perspectives ofkey stakeholders. In terms of statisticalmeasures used to analyze the effectsof using electronic voting or countingtechnologies, the following would be useful:• Turnout — A simple comparison ofturnout for pilot project polling stationsagainst the previous electoral turnoutin the polling stations is not a goodmeasure of the effect of using electronicvoting or counting technologies. Thecomparison would need to factor inthe type of election, as some electionshave more turnout than others, andthe general trends in turnout for similarpolling stations in non-pilot areas.Given such considerations, the turnoutexperienced at pilot project pollinglocations can provide some indicationof the possible effect on turnout causedby the use of electronic voting orcounting technology.• Speed of Voting — In order to fullyunderstand the consequences ofusing electronic voting and countingtechnologies it is important to collectdata about how quickly votersare able to cast their votes usingthese technologies. This will haveconsequences for the logistics andcosts of using electronic technologieson a wider scale. If using technologies isfaster or slower than the existing systemof balloting then this may require orallow changes in the number of votersallocated to a polling station. This datashould be collected by polling staffworking in pilot project locations.70Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT STANDARDS PROJECT• Speed of Results — The overall timeit takes from the close of polls to theannouncement of polling station resultsand the tabulation and announcementof results for the constituency should bemonitored. A comparison can be made tothe existing system of balloting in order toquantify any improvements in this regardwhen using electronic voting or countingtechnologies. If different electronicsolutions are being piloted then separatedata should be collected and reported foreach solution being used.• Complaints Received — The numberand severity of complaints receivedabout polling in pilot project areascompared to non-pilot project areascan be seen as a measure of theacceptability of using electronic votingand counting technologies.• Number of Blank Votes — While difficultto interpret, the number of blankballots cast using electronic voting orcounting technologies is an importantindicator. The interpretation of thisneeds to be assessed in parallel withhistorical statistics data on blank ballotsand overvotes (casting votes for morecandidates on a ballot than permitted),and also in the context of how manyballots the voter is being presented with.While electronic voting and countingsystems can be programmed not toallow overvotes, emerging standardsindicate that a blank ballot should bepossible. 104 If the number of blankballots is significantly different from104 Council of Europe (2004) Recommendation 13.the existing system of balloting, thenit is a consequence that needs tobe understood. It may be a positiveconsequence or a negative one.• Help Desk Logs — The number ofissues and questions received by thehelp desk will be an indicator of notonly how reliable the electronic votingor counting systems piloted were, butalso the kinds of problems that wereencountered in using the systems andimplementing the procedures writtenfor their use. The analysis of thisinformation may indicate that changes inthe solution or implementing procedureswould be required if adoption proceeds.• Results of the Mandatory Audit — It is tobe hoped that the mandatory audit willdemonstrate that the audited results werethe same as the electronic results. Shouldany differences have been found, this factwill need to be carefully considered inthe evaluation, and remedial action maybe required to fix these discrepanciesbefore proceeding in any way with furtherpiloting or adopting electronic voting orcounting technologies.• Election Related Violence — In somecountries the electoral process, andespecially the counting process, can betense events. Sometimes tensions lead toelection related violence as attempts aremade to manipulate the work of pollingand counting staff, or perception is rife thatsuch manipulation is taking place. Takingthe human element out of the countingprocess can diffuse these tensions.International Foundation for Electoral Systems 71

INTERNATIONAL PILOT PROJECTELECTORAL STANDARDSAnother indicator of the consequencesof using electronic technologies couldbe in the incident rate of electionrelated violence in pilot project locationscompared to non-pilot project locations.of the use of the electronic voting orcounting technology, and if so, why?Were potential voters sufficiently wellinformed about how to use the electronicvoting or counting solution?But these quantitative measures can onlyprovide so much information about the use ofelectronic voting and counting technologies.They need to be supplemented by qualitativeassessments of the following kinds of issuesand questions:• Voter Experience — Were voters awareof the pilot project before attendingthe polling location? Was sufficientinformation provided prior to the pilotto enable voters to understand howto use electronic voting or countingtechnologies? How easy did votersfind the electronic voting or countingtechnologies to use? Did they experiencespecific problems in using them? Wereinstructions clear? Were there any issueswith activation and ensuring completionof the voting process? Did the votersunderstand the process of review,confirmation and VVPAT (if there wasa VVPAT)? Did the electronic voting orcounting solution increase or decreasethe need for assisted voting? How couldthe process have been made easier andclearer? Did the voter cast a vote in allelections (if multiple races were beingpiloted), if not, then why? What was thereaction of people in marginalized groupssuch as illiterate and visually impaired?• Non-Voter Motivations — Did somevoters not participate in the pilot because• Configuration of Electronic Votingor Counting Hardware — Howcomprehensive were the instructionson configuring the electronic voting orcounting hardware? How easy wasthe process of configuration? Whatspecialized skills would be of benefit forthose configuring the electronic votingor counting hardware? How might theprocess of configuration be improved?• Polling Station Setup — Howcomprehensive were the instructions onsetting up electronic voting or countingmachines? How easy was the process ofsetting up electronic voting or countingmachines? What specialized skills wouldbe of benefit for those setting up themachines? How might the process ofsetting up the machines be improved?• Polling — How smoothly did theelectronic voting or counting systemoperate during polling? Were anyproblems encountered, and if so, whatwere they? What voter feedback wasreceived about using the electronicvoting or counting system? How long didit take on average for voters to cast theirballot using the system? Did voters seemto understand how to use the electronicsystem or did they often have to seekhelp from the polling staff?72Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL PILOT STANDARDS PROJECT• Close of Polls — How easy werethe procedures for the close of pollsusing the electronic voting or countingsolution? How comprehensive werethe instructions for the close of pollsusing the electronic solution? Wasthe solution able to produce electionresults efficiently?• Tabulation of Results — How effectivelydid the process of transfer/transmissionof results from the electronic votingor counting system to the centralcount tabulation take place? Did theprocedures adequately provide forthe process? Could they have beenimproved in any way? How did theresults tabulation software perform?Were there improvements in the speed,accuracy and quality of results tabulationprocedures and outputs?• Manual Audit — How well did the audittrail function? How easy was it to manuallycount the audit trails? Was the audit trailkept in a way that the secrecy of the votewas maintained both during voting and themanual count of the audit trail?• Electoral Roll — If the electronic voting orcounting solution included an electronicversion of the electoral roll, how smoothwas the process of identification andverification? Was the process faster orslower than using a paper voter list? Wasenfranchisement increased/decreased?Were there fewer or more challenges tovoter identity by political agents than incomparable previous elections or otherpolling locations?• Logistics — To what extent did thelogistical plan for distribution, security,retrieval and storage of the electronicvoting or counting hardware and auditmechanisms hold up? Were thereunforeseen problems with breakage,batteries, theft, maintenance, etc.?Was the manpower sufficient inquantity and quality?• Security — Were the security measuresrelated to the use of the electronicvoting and counting technology seenas sufficient to protect the integrity andaccuracy of the process? Are thererecommendations for improvements inthese security measures for the futureimplementation of such technology?• Observation of the Process — Whatwere the opinions of observers andcandidate/party agents about the useof the electronic voting or countingtechnology? Did observers and agentsbelieve that the technology improved theprocess of balloting and counting? Wereobservers and agents able to observekey aspects of the process whereelectronic technologies were used? Didthe polling officials demonstrate to theobservers and agents that the electronicballot box was empty at the start ofpolling, and did this provide a feeling ofassurance that this was the case?• Comparison to Existing System ofBalloting — To what extent was theuse of electronic voting or countingtechnologies better or worse than thecurrent system of balloting?International Foundation for Electoral Systems 73

PILOT PROJECTThis qualitative data about the use ofelectronic voting and counting technologiesin the pilot project can be collected througha number of means. Interviews can beconducted with a sample of the variousgroups whose opinions will be critical tothe assessment — voters, non-voters,staff, observers, candidates and partyrepresentatives. A sample of voters can beinterviewed as they leave the pilot projectlocations in order to obtain their immediateimpressions of using the electronic votingor counting solutions. In addition, focusgroups can be used to explore the opinionsof stakeholders to some of the key findingsfrom an initial assessment of the pilot.Focus groups can also be used to try andexplore the reasons for unexpected opinionsconsistently expressed in interviews or toexplain anomalous statistical data collectedduring the pilot (for example, high or lowlevels of blank ballots or the time it takes tocast a ballot using the electronic voting orcounting solution).All of these means of evaluating the pilotproject should be utilized in order to obtainas complete a picture as possible of theoperation of the electronic voting or countingtechnology during the pilot and the reactionsof the key stakeholders to using this solution.This evaluation should be written up into aPilot Project Report covering the process ofconducting the pilot project, the conclusionsand recommended next steps with respect toimplementing electronic voting and countingtechnologies (Annex 5 shows a sample tableof contents for an evaluation report).74Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

The Decision on AdoptionThe result of the pilot project may be to adopt a new voting and countingtechnology system, not to adopt it or to conduct further tests.The Pilot Project Report will need to be carefully reviewed by the Feasibility StudyCommittee, if it is different from the Pilot Project Committee. The Feasibility StudyCommittee may decide to accept, reject or amend the conclusions and recommendations ofthe Pilot Project Report.A number of general conclusions and next steps may be reached as a result of the pilot project:• Not Proceed with Electronic Voting and Counting Technologies — It may be decided thatelectronic voting and counting technologies either do not meet the needs of the electionsin question, or they do meet the needs but the benefits to be gained do not justify theexpenditure required to implement them or the disruption caused by implementing them.In either case it will be important to clearly identify the reasons why the recommendationis made to not proceed. This will be important in the future. If cost, functionality or ease ofimplementing the technologies change, then this recommendation can be easily revisited.• Additional Piloting — For a number of reasons it may be decided that a recommendationcannot be made to proceed with the implementation of electronic voting and countingtechnologies, but also that investigation into their use should not be ended.It may be that the original specification developed for the electronic voting and countingtechnologies was defective or insufficient and the solutions with different functionalityor features would be better suited to the electoral environment. It may be that in theInternational Foundation for Electoral Systems 75

INTERNATIONAL THE DECISION ON ELECTORAL ADOPTIONSTANDARDSfinal analysis the electronic voting orcounting solutions provided did notproperly meet the specification. Thepilot project report may concludethat voter education was insufficientor the procedures used during thepilot were not adequate. Any of theseconclusions would indicate that thepiloting of electronic voting and countingtechnologies should continue, as longas the anticipated benefits were stilljustified by the previous pilot findings.The initial pilot may also have been ona very small scale. Even if the resultswere very positive it may be decidedthat before a recommendation is madeto move towards full implementationthe pilot needs to be repeated, withan expanded scale and scope in orderto better test the electronic voting orcounting solution. In fact, it makessense to pilot electronic voting andcounting technologies on multipleoccasions before moving ahead with afull scale implementation.• Adoption of Electronic Voting or CountingTechnologies — If the pilot project wassuccessful, demonstrating that electronicvoting or counting technologies workedeffectively and delivered significantbenefits to the electoral process, thenthe recommendation may be to proceedwith the full scale implementation of thetechnology. As indicated above, such arecommendation should not be basedon a single, small scale pilot, but on thesuccessful conduct of a series of pilotsor a single large scale pilot.“Even where therecommendation is to movetowards the full adoptionof electronic voting andcounting technologies, therecommendation may be tomove towards this adoption ina staggered manner.”Should the adoption of electronicvoting or counting technologies berecommended, it is still important torecognize that there may be importantlessons to learn from the pilot project.Time must be provided so that lessonsfrom the pilot can be properly adaptedbefore the adoption of electronic votingor counting technologies. This mayrequire technical specifications, pollingand counting procedures, trainingplans and voter education schemesto be reconsidered and redrafted.The procurement process will mostlikely have to start anew; given thepotential changes and the larger sizeof the contract for electronic voting orcounting products. Failure to learn fromthe pilot, however, could have seriousimplications for the success of the largerscale adoption of electronic voting andcounting technologies.Even where the recommendation isto move towards the full adoptionof electronic voting and countingtechnologies, the recommendationmay be to move towards this adoptionin a staggered manner, as other76Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL THE DECISION ELECTORAL ON STANDARDSADOPTIONcountries have done. 105 Such staggeredadoption of electronic voting andcounting technologies may make agreat deal of sense as it allows for thefinancial burden to be spread overseveral budget cycles. However, suchstaggered implementation may also beproblematic as it entails fundamentaldifferences in the way in which votingrights are applied for different voters.solution against the opposition of all or most ofthe key stakeholders in the process.Once the Feasibility Study Report has beenfinalized, after this consultative process, thefull report should be made public and the mainrecommendations issued through a pressrelease by the Feasibility Study Committee.At this stage of the process theserecommendations should only be consideredas preliminary. In the interests of opennessand transparency it is important that thesepreliminary recommendations be subjectto consultation with key stakeholders. Theconsultation process should be used toexplain the details of the pilot project tostakeholders, the conclusions reached andthe recommendations being made withrespect to the adoption of electronic voting orcounting technologies.It is to be hoped that this consultation processwill complement feedback previously receivedby stakeholders throughout the process,but this may not be the case. Should theopinions of stakeholders be consistentlyopposed to the recommendations of theFeasibility Study Committee, then the causesand consequences of such disagreement willneed to be carefully considered. It would be abrave, possibly foolhardy, EMB that proceededwith adopting an electronic voting or counting105 See the example of India. Indian electronic votingmachines were first piloted in 1982 in a limitednumber of polling stations for a by-election andwas finally used country wide for general electionsin 2004 - http://pib.nic.in/elections2009/volume1/Chap-39.pdf (last accessed on 31 January 2011).International Foundation for Electoral Systems 77

78Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

Additional ResourcesThis list of materials on electronic voting and counting technologies in no way represents acomprehensive list of all of the materials available. However, it represents a good startingpoint for a good understanding of issues related to the use of such technologies for theconduct of elections.Braun, N. and Brandli, D. (2006) Swiss E-Voting Pilot Projects: Evaluation, Situation Analysisand How To ProceedBrennan Center for Justice at New York University School of Law (2006) The Machinery ofDemocracy: Protecting Elections in an Electronic World, Brennan Center Task Forceon Voting System Security.Caarls, S. (2010) E-voting Handbook: Key steps in the implementation of e-enabled elections,Council of Europe Publishing: StrasbourgCentre for Human Rights (1994) Professional Training Series No.2: Human Rights andElections — A Handbook on the Legal Technical and Human Rights Aspects ofElections, United Nations: New York and GenevaCouncil of Europe (2004) Legal, Operational and Technical Standards for E-Voting,Recommendation Rec(2004)11 adopted by the Committee of Ministers of the Councilof Europe on 30 September 2004 and Explanatory MemorandumCouncil of Europe (2010) E-Voting Handbook: Key steps in the implementation of e-enabledelectionsElection Commission of Pakistan’s (2010) Committee on the Use of Electronic VotingMachines in Pakistan: Final Report and RecommendationEuropean Commission (2006) Methodological Guide to Electoral AssistanceEuropean Commission (2007) Compendium of International Electoral Standards: SecondEdition, European Commission: BrusselsEuropean Commission for Democracy Through Law (Venice Commission) (2002) Code ofGood Practice in Electoral Matters: Guidelines and Explanatory Report, Adoptedby the Venice Commission at its 52 nd session (Venice, 18-19 October 2002), CDL-AD(2002) 23 revGoodwin-Gill, G. (1994) Free and Fair Elections: International Law and Practice, Inter-Parliamentary Union: GenevaGoodwin-Gill, G. (2006) Free and Fair Elections: New Expanded Edition, Inter-ParliamentaryUnion: GenevaGovernment of Buenos Aires (2005) 2005 E-Voting Pilot Project: Preliminary Report of ResultsInternational Foundation for Electoral Systems 79

ADDITIONAL RESOURCESIFES Applied Research Center (2007) Challenging the Norms and Standards of ElectionAdministrationIrish Commission on Electronic Voting (2006) Second Report of the Commission on ElectronicVoting on the Secrecy, Accuracy and Testing of the Chosen Electronic Voting System.Joint EC-UNDP Task Force on Electoral Assistance (2010) Procurement Aspects ofIntroducing ICT Solutions in Electoral Processes: The Specific Case of VoterRegistrationKrimmer, R. (Ed.) (2006) Electronic Voting 2006: Overview of Proceedings of 2 nd InternationalWorkshop, co-organised by the Council of Europe, ESF-TED, IFIP WG8.6 andE-Voting.CCNorwegian Ministry of Local Government and Regional Development (2006) Electronic voting— challenges and opportunitiesOAS (2010) Observing the Use of Electoral Technologies: A Manual for OAS ElectoralObservation Missions, General Secretariat of the Organization of American States(GS/OAS)Open Rights Group (2007) May 2007 Election Report: Findings of the Open Rights GroupElection Observation Mission in Scotland and England.OSCE (2005) Challenges of Election Technologies and Procedures: Final Report,Supplementary Human Dimension Meeting, PC.SHDM.GAL/5/05OSCE (2007) Election Observation Handbook: Fifth Edition, OSCE Office for DemocraticInstitutions and Human Rights: WarsawOSCE (2008) OSCE/ODIHR Discussion Paper in Preparation of Guidelines for the Observationof Electronic Elections, ODIHR.GAL/73/08Pran, V. and Merloe, P. (2007) Monitoring Electronic Technologies in Electoral Processes: AnNDI Guide for Political Parties and Civic Organizations, National Democratic Institutefor International AffairsThe Carter Center (2007) Developing a Methodology for Observing Electronic VotingTokaji, D. (2004) The Paperless Chase: Electronic Voting and Democratic Values, MoritzCollege of Law, Ohio State UniversityUK Election Commission (2007) Key Issues and Conclusions: May 2007 electoral pilotschemesUS Election Assistance Commission’s (2005) Voluntary Voting System GuidelinesYard, M. (ed.) (2010) Direct Democracy: Progress and Pitfalls of Election Technology80Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

AnnexesAnnex 1 — Draft Timeline for Decision in Principle 82Annex 2 — Feasibility Study Committee Decision in Principle ReportTable of Contents 84Annex 3 — Suggested Briefing Pack for Feasibility Study Committee 85Annex 4 — Example of Weighted Ranking of Vendor Proposals 86Annex 5 — Sample Pilot Project Report Table of Contents 87International Foundation for Electoral Systems 81

ANNEXESINTERNATIONAL ELECTORAL STANDARDSAnnex 1 - Draft Timeline for Decision in PrincipleThe timeline will depend a lot on the amount of time that the Feasibility Study Committee andworking groups can allocate to the issue. A full-time Feasibility Study Committee will deal withthe issue much quicker.ID Task Name Duration Start Finish1 Mandate of Feasibility Study Established 1 day Mon 03/01/11Mon 03/01/112 Feasibility Study Committee 18 days Thu 13/01/11Mon 27/06/113 First Meeting of the Committee 1 wk Thu 13/01/11 Fri 21/01/114 Monthly Committee Meetings 86 days Tue 01/02/11 Tue 31/05/115 Development of Findings and Recommendations 1 day Tue 07/06/11 Tue 07/06/116 Finalisation of Report 2 wks Wed 08/06/11Mon 27/06/117 Obtaining Vendor Information 50 days Mon 24/01/11 Fri 01/04/118 Establish Vendor Point of Contact 1 day Mon 24/01/11Mon 24/01/119 Information Request to Vendors 2 wks Thu 03/02/11 Tue 22/02/1110 Revised Information Request to Vendors 2 wks Tue 15/03/11 Fri 01/04/1111 Working Group Issue Consideration 78 days Mon 24/01/11Wed 11/05/1112 1 - Assessment of Current System 2 wks Mon 24/01/11 Thu 10/02/1113 2 - Assessment of New Technologies 2 wks Wed 23/02/11Mon 14/03/1114 3 - Security Aspects 2 wks Tue 15/03/11 Fri 01/04/1115 4 - Determining Technical Feasibility 2 wks Mon 04/04/11 Thu 21/04/1116 5 - Cost Benefit Analysis 2 wks Fri 22/04/11Wed 11/05/1117 6 - Institutional Capacity 2 wks Fri 22/04/11Wed 11/05/1118 7 - Legal Reform 2 wks Fri 22/04/11Wed 11/05/1119 Study Trips 15 days Fri 22/04/11Thu 12/05/1120 Study Trip 1 1 wk Fri 22/04/11Mon 02/05/1121 Study Trip 2 1 wk Tue 03/05/11Wed 11/05/1122 Report Back to Committee 1 day Thu 12/05/11 Thu 12/05/1123 Vendor Demonstration 78 days Tue 25/01/11Thu 12/05/1124 Identify Supliers to Invite 1 wk Tue 25/01/11Wed 02/02/1125 Issue Invitations to Vendors 1 day Thu 03/02/11 Thu 03/02/1126 Timeline for Acceptance of Invitation 1 mon Fri 04/02/11 Fri 18/03/1127 Issue Visa Support Letters 1 day Mon 21/03/11Mon 21/03/1128 Vendor Demonstration 1 day Thu 12/05/11 Thu 12/05/1129 Stakeholder Consultation 3 days Tue 24/05/11 Thu 26/05/11January February M27/12 03/01 10/01 17/01 24/01 31/01 07/02 14/02 21/02 2Project: IFES Voting and Counting TecDate: Thu 07/04/11TaskSplitProgressMilestoneSummaryProject SummaryExternal TasksExternal MilestoneDeadlinePage 182Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL STANDARDSANNEXESMarch April May June July28/02 07/03 14/03 21/03 28/03 04/04 11/04 18/04 25/04 02/05 09/05 16/05 23/05 30/05 06/06 13/06 20/06 27/06 04/07Project: IFES Voting and Counting TecDate: Thu 07/04/11TaskSplitProgressMilestoneSummaryProject SummaryExternal TasksExternal MilestoneDeadlinePage 2International Foundation for Electoral Systems 83

ANNEXESINTERNATIONAL ELECTORAL STANDARDSAnnex 2 — Decision in Principle Report Table of ContentsTABLE OF CONTENTSA. Executive SummaryB. IntroductionBackground for the Feasibility StudyMandate of the Feasibility Study CommitteeWorking Arrangements for the Work of the CommitteeC. Working Group Findings and RecommendationsIssue 1 — Assessment of the Current System of Paper BallotingIssue 2 — Assessment of the Advantages and Disadvantages Offered by Voting andCounting TechnologiesIssue 3 — Review of IT Security AspectsIssue 4 — Determining Technical FeasibilityIssue 5 — Cost Benefit AnalysisIssue 6 — Institutional CapacityIssue 7 — Legal Reform IssuesD. Findings and Recommendations from Study Trips ConductedE. Vendor Demonstration and Stakeholder ConsultationsF. Findings of the Feasibility Study CommitteeG. Recommendations and Next StepsH. AnnexesRequirements for Electronic Voting and Counting TechnologiesDraft Mandate for the Conduct of a Pilot ProjectDraft Budget for the Conduct of a Pilot Project84Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL STANDARDSANNEXESAnnex 3 — Suggested Briefing Pack for Feasibility StudyCommitteeGeneral Documents on Electronic Voting and Counting Technologies1. Council of Europe (2004) Legal, Operational and Technical Standards for E-Voting,Recommendation Rec (2004)11 adopted by the Committee of Ministers of the Council ofEurope on 30 September 2004, and Explanatory Memorandum.2. Council of Europe (2010) E-Voting Handbook: Key steps in the implementation ofe-enabled elections.3. The Carter Center (2007) Developing a Methodology for Observing Electronic Voting.4. OSCE/ODIHR (2008) OSCE/ODIHR Discussion Paper: In Preparation of Guidelines for theObservation of Electronic Voting.5. IFES Applied Research Center (2007) Challenging the Norms and Standards of ElectionAdministration.Country Experiences of Using Electronic Voting and Counting Technologies1. Brennan Center for Justice at New York University School of Law (2006) The Machineryof Democracy: Protecting Elections in an Electronic World, Brennan Center Task Force onVoting System Security.2. Tokaji, D. (2004) The Paperless Chase: Electronic Voting and Democratic Values, MoritzCollege of Law, Ohio State University.3. Open Rights Group (2007) May 2007 Election Report: Findings of the Open Rights GroupElection Observation Mission in Scotland and England.4. Irish Commission on Electronic Voting (2006) Second Report of the Commission onElectronic Voting on the Secrecy, Accuracy and Testing of the Chosen Electronic VotingSystem.International Foundation for Electoral Systems 85

ANNEXESINTERNATIONAL ELECTORAL STANDARDSAnnex 4 — Example of Weighted Ranking of VendorProposalsSample General Evaluation CriteriaEvaluation Criteria Minimum Rating WeightingCompliance with technical specifications 7 out of 10 50%Experience of the vendor 5 out of 10 15%Quality of the project management team - 10%Access to source code 10%Price 5 out of 10 15%Example of Applying These CriteriaProposal 1 assessment:Evaluation Criteria Rating (out of 10) Meet Minimum? Weighted ScoreCompliance with technical specifications 7 50% 35Experience of the vendor 8 15% 12Quality of the project management team 3 10% 3Access to source code 0 10% 0Price 8 15% 12Total proposal score 62Proposal 2 assessment:Evaluation Criteria Rating (out of 10) Meet Minimum? Weighted ScoreCompliance with technical specifications 9 50% 45Experience of the vendor 3 15% 4.5Quality of the project management team 5 10% 5Access to source code 3 10% 3Price 2 15% 3Total proposal score 60.5In this example, although Proposal 2 is the most compliant with the technical specificationsfor the system, Proposal 1 is ranked higher because of other evaluation criteria.86Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

INTERNATIONAL ELECTORAL STANDARDSANNEXESAnnex 5 — Sample Pilot Project Report Table of ContentsTABLE OF CONTENTSA. Executive SummaryB. IntroductionBackground for the Pilot ProjectMandate of the Pilot ProjectSelection of Pilot Project LocationsC. Management Arrangements for the Pilot ProjectPilot Project CommitteePilot Project Manager and Supporting ResourcesRisk Management StrategyD. Procurement Process for Electronic Voting and Counting MachinesTendering ProcessEvaluation of BidsSelection of VendorContracting of VendorDelivery ScheduleE. Testing and CertificationElection Management Body TestingIndependent Testing and Certification ProcessF. Procedural FrameworkLegislative Changes Required By Use of Electronic Voting and Counting TechnologiesRegulatory Changes RequiredProcedural DevelopmentsG. Implementing the PilotTraining of StaffVoter EducationConsulting With StakeholdersTraining of Observers and Political Party/Candidate AgentsTechnical Support and Help Desk ServicesElection Day OperationsAudit of ResultsInternational Foundation for Electoral Systems 87

ANNEXESH. Evaluation of the Pilot ProjectProject Management EvaluationElection Management Body Staff FeedbackStatistical Data CollectedFeedback from Voters and Non VotersFeedback from Other Electoral StakeholdersEvaluation of Technology Solution Used in the PilotAssessment of Anticipated Benefits Resulting from the Pilot TechnologiesAssessment of Anticipated Challenges Resulting from the Pilot TechnologiesEvaluation of Cost of Implementing the Pilot TechnologiesI. Findings of the Pilot ProjectJ. Recommendations and Next Steps88Electronic Voting & Counting Technologies: A Guide to Conducting Feasibility Studies

ELECTION TECHNOLOGY SERIESIFES 1850 K Street NW, 5 th Floor • Washington, DC 20006 | www.IFES.org