Lecture 8: Verification and Validation

Size: px
Start display at page:

Download "Lecture 8: Verification and Validation"

Transcription

1 Thanks to Prof. Steve Easterbrook University of Toronto What are goals of V&V Validation Techniques Ø Inspection Ø Model Checking Ø Prototyping Verification Techniques Ø Consistency Checking Lecture 8: Verification and Validation Ø Making Specifications Traceable Independent V&V 278

2 Thanks to Prof. Steve Easterbrook University of Toronto What are goals of V&V V&V Techniques Ø Inspection Ø Model Checking Ø Prototyping Lecture 8: Verification and Validation Ø Making Specifications Traceable Ø Consistency Checking Independent V&V 279

3 Verification and Validation Validation: Ø Are we building the right system? Ø Does our problem statement accurately capture the real problem? Problem Situation Ø Did we account for the needs of all the stakeholders? Verification: Ø Are we building the system right? Ø Does our design meet the spec? Ø Does our implementation meet the spec? Ø Does the delivered system do what we said it would do? Verification Problem Statement Implementation Statement Validation Ø Are our requirements models consistent with one another? System 280

4 Application Domain Refresher: V&V Criteria Machine Domain Some distinctions: Ø Domain Properties: things in the application domain that are true anyway Ø Requirements: things in the application domain that we wish to be made true Ø Specification: a description of the behaviours the program must have in order to meet the requirements Two verification criteria: Ø The Program running on a particular Computer satisfies the Specification Ø The Specification, given the Domain properties, satisfies the Requirements Two validation criteria: Ø Did we discover (and understand) all the important Requirements? Ø Did we discover (and understand) all the relevant Domain properties? 281

5 Example: Ø Requirement R: V&V Example Ø Reverse thrust shall only be enabled when the aircraft is moving on the runway Ø Domain Properties D: Ø Wheel pulses on if and only if wheels turning Ø Wheels turning if and only if moving on runway Ø Specification S: Verification Ø Reverse thrust enabled if and only if wheel pulses on Ø Does the flight software, P, running on the aircraft flight computer, C, correctly implement S? Ø Does S, in the context of assumptions D, satisfy R? Validation Ø Are our assumptions, D, about the domain correct? Did we miss any? Ø Are the requirements, R, what is really needed? Did we miss any? 282

6 Inquiry Cycle Prior Knowledge (e.g. customer feedback) Initial hypotheses Intervene (replace the old system) Carry out the experiments (manipulate the variables) Observe (what is wrong with the current system?) Look for anomalies - what can t the current theory explain? Design experiments to test the new theory Design (invent a better system) Model (describe/explain the observed problems) Create/refine a better theory 283

7 Shortcuts in the inquiry cycle Prior Knowledge (e.g. customer feedback) Observe (what is wrong with the the current the prototype?) model?) system?) Check properties of the model Intervene (replace the old system) Get users to try it Analyze the model Model (describe/explain the observed problems) Build a Prototype Design (invent a better system) 284

8 Thanks to Prof. Steve Easterbrook University of Toronto What are goals of V&V V&V Techniques Ø Inspection Ø Model Checking Ø Prototyping Lecture 8: Verification and Validation Ø Making Specifications Traceable Ø Consistency Checking Independent V&V 285

9 Reviews, Walkthroughs, Inspections Management reviews Ø E.g. preliminary design review (PDR), critical design review (CDR), Ø Used to provide confidence that the design is sound Ø Attended by management and sponsors (customers) Ø Often just a dog-and-pony show Walkthroughs Ø developer technique (usually informal) Ø used by development teams to improve quality of product Ø focus is on finding defects These definitions are not widely agreed! Ø Other terms used: Ø Formal Technical Reviews (FTRs) Ø Formal Inspections Formality can vary: Ø informal: Ø meetings over coffee, Ø regular team meetings, Ø etc. Ø formal: Ø scheduled meetings, Ø prepared participants, Ø defined agenda, Ø specific format, Ø documented output 286

10 Benefits of formal inspection Formal inspection works well for programming: Ø For applications programming: Ø more effective than testing Ø most reviewed programs run correctly first time Ø Data from large projects Ø error reduction by a factor of 5; (10 in some reported cases) Ø improvement in productivity: 14% to 25% Ø percentage of errors found by inspection: 58% to 82% Ø cost reduction of 50%-80% for V&V (even including cost of inspection) Ø Effects on staff competence: Ø increased morale, reduced turnover Ø better estimation and scheduling (more knowledge about defect profiles) Ø better management recognition of staff ability These benefits also apply to requirements inspections Ø Many empirical studies investigated variant inspection processes Ø Mixed results on the relative benefits of different processes 287

11 Model Checking Has revolutionized formal verification: Ø emphasis on partial verification of partial models Ø E.g. as a debugging tool for state machine models Ø fully automated What it does: Ø Mathematically computes the satisfies relation: Ø Given a temporal logic theory, checks whether a given finite state machine is a model for that theory. Ø Engineering view checks whether properties hold: Ø Given a model (e.g. a FSM), checks whether it obeys various safety and liveness properties How to apply it in RE: Ø The model is an (operational) Specification Ø Check whether particular requirements hold of the spec Ø The model is (an abstracted portion of) the Requirements Ø Carry out basic validity tests as the model is developed Ø The model is a conjunction of the Requirements and the Domain Ø Formalise assumptions and test whether the model respects them 288

12 Prototyping A software prototype is a partial implementation constructed primarily to enable customers, users, or developers to learn more about a problem or its solution. [Davis 1990] Prototyping is the process of building a working model of the system [Agresti 1986] Approaches to prototyping Ø Presentation Prototypes Ø explain, demonstrate and inform then throw away Ø e.g. used for proof of concept; explaining design features; etc. Ø Exploratory Prototypes Ø used to determine problems, elicit needs, clarify goals, compare design options Ø informal, unstructured and thrown away. Ø Breadboards or Experimental Prototypes Ø explore technical feasibility; test suitability of a technology Ø Typically no user/customer involvement Ø Evolutionary (e.g. operational prototypes, pilot systems ): Ø development seen as continuous process of adapting the system Ø prototype is an early deliverable, to be continually improved. 289

13 Throwaway or Evolve? Throwaway Prototyping Ø Purpose: Ø to learn more about the problem or its solution Ø discard after desired knowledge is gained. Ø Use: Ø early or late Ø Approach: Ø horizontal - build only one layer (e.g. UI) Ø quick and dirty Ø Advantages: Ø Learning medium for better convergence Ø Early delivery early testing less cost Ø Successful even if it fails! Ø Disadvantages: Ø Wasted effort if reqts change rapidly Ø Often replaces proper documentation of the requirements Ø May set customers expectations too high Ø Can get developed into final product Evolutionary Prototyping Ø Purpose Ø to learn more about the problem or its solution Ø and reduce risk by building parts early Ø Use: Ø incremental; evolutionary Ø Approach: Ø vertical - partial impl. of all layers; Ø designed to be extended/adapted Ø Advantages: Ø Requirements not frozen Ø Return to last increment if error is found Ø Flexible(?) Ø Disadvantages: Ø Can end up with complex, unstructured system which is hard to maintain Ø early architectural choice may be poor Ø Optimal solutions not guaranteed Ø Lacks control and direction 290

14 Thanks to Prof. Steve Easterbrook University of Toronto What are goals of V&V V&V Techniques Ø Inspection Ø Model Checking Ø Prototyping Lecture 8: Verification and Validation Ø Making Specifications Traceable Ø Consistency Checking Independent V&V 291

15 We ve looked at the following non-uml diagrams Ø Goal Models Ø Capture strategic goals of stakeholders Ø Good for exploring how and why questions with stakeholders Ø Good for analysing trade-offs, especially over design choices Ø Strategic Dependency Models (i*) Ø Capture relationships between actors in an organisational setting Ø Helps to relate goal models to organisational setting Ø Good for understanding how the organisation will be changed 292

16 Class diagrams Ø Class Diagrams Ø capture the structure of the information used by the system Ø good for analysing the relationships between data items used by the system Ø good for helping you identify a modular structure for the system Ø Cross checks Ø Does the class diagram capture all the classes mentioned in other diagrams? Ø Does every class have methods to get/set its attributes? 293

17 Use cases Ø Use Cases Ø capture the view of the system from the view of its users Ø good starting point for specification of functionality Ø good visual overview of the main functional requirements Ø Cross-checks: Ø Does each use case have a user? Does each user have at least one use case? Ø Is each use case documented? Using sequence diagrams or use case template 294

18 Statecharts Ø Statecharts Ø capture all possible responses of an object to all uses cases in which it is involved Ø good for modeling the dynamic behavior of a class of objects Ø good for analyzing event ordering, reachability, deadlock, etc. Cross-checks: Ø Does each statechart diagram capture (the states of) a single class? Ø Is that class in the class diagram? Ø Does each transition have a trigger event? Ø Is it clear which object initiates each event? Ø Is each event listed as an operation for that object s class in the class diagram? Ø Does each state represent a distinct combination of attribute values? Ø Is it clear which combination of attribute values? Ø Are all those attributes shown on the class diagram? Ø Are there method calls in the class diagram for each transition? Ø a method call that will update attribute values for the new state? Ø method calls that will test any conditions on the transition? Ø method calls that will carry out any actions on the transition? 295

19 Ø Sequence Diagrams Sequence diagrams Ø capture an individual scenario (one path through a use case) Ø good for modelling dialog structure for a user interface or a business process Ø good for identifying which objects (classes) participate in each use case Ø helps you check that you identified all the necessary classes and operations Ø Cross-checks: Ø Is each class in the class diagram? Ø Can each message be sent? Is there an association connecting sender and receiver classes on the class diagram? Is there a method call in the sending class for each sent message? Is there a method call in the receiving class for each received message? 296

20 Verification Model Analysis Ø Is the model well-formed? Ø Are the parts of the model consistent with one another? Validation: Ø Animation of the model on small examples Ø Formal challenges: Ø if the model is correct then the following property should hold... Ø What if questions: Ø reasoning about the consequences of particular requirements; Ø reasoning about the effect of possible changes Ø will the system ever do the following... Ø State exploration Ø E.g. use a model checking to find traces that satisfy some property 297

21 Thanks to Prof. Steve Easterbrook University of Toronto What are goals of V&V V&V Techniques Ø Inspection Ø Model Checking Ø Prototyping Lecture 8: Verification and Validation Ø Making Specifications Traceable Ø Consistency Checking Independent V&V 298

22 Independent V&V V&V performed by a separate contractor Ø Independent V&V fulfills the need for an independent technical opinion. Ø Cost between 5% and 15% of development costs Ø Studies show up to fivefold return on investment: Ø Errors found earlier, cheaper to fix, cheaper to re-test Ø Clearer specifications Ø Developer more likely to use best practices Three types of independence: Ø Managerial Independence: Ø separate responsibility from that of developing the software Ø can decide when and where to focus the V&V effort Ø Financial Independence: Ø Costed and funded separately Ø No risk of diverting resources when the going gets tough Ø Technical Independence: Ø Different personnel, to avoid analyst bias Ø Use of different tools and techniques 299

23 Thanks to Prof. Steve Easterbrook University of Toronto What are goals of V&V Validation Techniques Ø Inspection Ø Model Checking Ø Prototyping Verification Techniques Ø Consistency Checking Lecture 8: Verification and Validation Ø Making Specifications Traceable Independent V&V 300

Key Considerations for Implementing Bodies and Oversight Actors

Key Considerations for Implementing Bodies and Oversight Actors Implementing and Overseeing Electronic Voting and Counting Technologies Key Considerations for Implementing Bodies and Oversight Actors Lead Authors Ben Goldsmith Holly Ruthrauff This publication is made

More information

Overview. Ø Neural Networks are considered black-box models Ø They are complex and do not provide much insight into variable relationships

Overview. Ø Neural Networks are considered black-box models Ø They are complex and do not provide much insight into variable relationships Neural Networks Overview Ø s are considered black-box models Ø They are complex and do not provide much insight into variable relationships Ø They have the potential to model very complicated patterns

More information

30 Transformational Design with Essential Aspect Decomposition: Model-Driven Architecture (MDA)

30 Transformational Design with Essential Aspect Decomposition: Model-Driven Architecture (MDA) Fakultät Informatik, Institut für Software- und Multimediatechnik, Lehrstuhl für Softwaretechnologie 30 Transformational Design with Essential Aspect Decomposition: Model-Driven Architecture () Prof. Dr.

More information

Aspect Decomposition: Model-Driven Architecture (MDA) 30 Transformational Design with Essential. References. Ø Optional: Ø Obligatory:

Aspect Decomposition: Model-Driven Architecture (MDA) 30 Transformational Design with Essential. References. Ø Optional: Ø Obligatory: Fakultät Informatik, Institut für Software- und Multimediatechnik, Lehrstuhl für Softwaretechnologie 30 Transformational Design with Essential Aspect Decomposition: Model-Driven Architecture () Prof. Dr.

More information

NEWSLETTER MESSAGE FROM DEAN VOTING SYSTEMS ASSESSMENT PROJECT IN THIS ISSUE FUNDING UPDATE JUNE 2015 VOL. 1 ISSUE 1

NEWSLETTER MESSAGE FROM DEAN VOTING SYSTEMS ASSESSMENT PROJECT IN THIS ISSUE FUNDING UPDATE JUNE 2015 VOL. 1 ISSUE 1 NEWSLETTER JUNE 2015 VOL. 1 ISSUE 1 MESSAGE FROM DEAN IN THIS ISSUE Message from Dean Engineering Kickoff The Agile Process and System Engineering User Testing Research Committee Events In the News Future

More information

30 Transformational Design with Essential Aspect Decomposition: Model-Driven Architecture (MDA)

30 Transformational Design with Essential Aspect Decomposition: Model-Driven Architecture (MDA) Fakultät Informatik, Institut für Software- und Multimediatechnik, Lehrstuhl für Softwaretechnologie 30 Transformational Design with Essential Aspect Decomposition: Model-Driven Architecture () Prof. Dr.

More information

Key Considerations for Oversight Actors

Key Considerations for Oversight Actors Implementing and Overseeing Electronic Voting and Counting Technologies Key Considerations for Oversight Actors Lead Authors Ben Goldsmith Holly Ruthrauff This publication is made possible by the generous

More information

Agent Modeling of Hispanic Population Acculturation and Behavior

Agent Modeling of Hispanic Population Acculturation and Behavior Agent of Hispanic Population Acculturation and Behavior Agent Modeling of Hispanic Population Acculturation and Behavior Lyle Wallis Dr. Mark Paich Decisio Consulting Inc. 201 Linden St. Ste 202 Fort Collins

More information

THE PRIMITIVES OF LEGAL PROTECTION AGAINST DATA TOTALITARIANISMS

THE PRIMITIVES OF LEGAL PROTECTION AGAINST DATA TOTALITARIANISMS THE PRIMITIVES OF LEGAL PROTECTION AGAINST DATA TOTALITARIANISMS Mireille Hildebrandt Research Professor at Vrije Universiteit Brussel (Law) Parttime Full Professor at Radboud University Nijmegen (CS)

More information

IDENTIFYING FAULT-PRONE MODULES IN SOFTWARE FOR DIAGNOSIS AND TREATMENT USING EEPORTERS CLASSIFICATION TREE

IDENTIFYING FAULT-PRONE MODULES IN SOFTWARE FOR DIAGNOSIS AND TREATMENT USING EEPORTERS CLASSIFICATION TREE IDENTIFYING FAULT-PRONE MODULES IN SOFTWARE FOR DIAGNOSIS AND TREATMENT USING EEPORTERS CLASSIFICATION TREE Bassey. A. Ekanem 1, Nseabasi Essien 2 1 Department of Computer Science, Delta State Polytechnic,

More information

Transition document Transition document, Version: 4.1, October 2017

Transition document Transition document, Version: 4.1, October 2017 Transition document Transition document, Version: 4.1, October 2017 Transition from a HACCP certification to a FSSC 22000 certification 1 Introduction... 2 2 General requirements for a transition to FSSC

More information

Trusted Logic Voting Systems with OASIS EML 4.0 (Election Markup Language)

Trusted Logic Voting Systems with OASIS EML 4.0 (Election Markup Language) April 27, 2005 http://www.oasis-open.org Trusted Logic Voting Systems with OASIS EML 4.0 (Election Markup Language) Presenter: David RR Webber Chair OASIS CAM TC http://drrw.net Contents Trusted Logic

More information

CHE 572: Modelling Process Dynamics

CHE 572: Modelling Process Dynamics Winter 2011 Instructor: Dr. J. Fraser Forbes office: ECERF 7-022 phone: (780) 492-0873 email: fraser.forbes@ualberta.ca office hours: Most days TA: office: email: Ms. Leily Mohammadi NREF 4 th Floor leily@ualberta.ca

More information

GRADE 9: Canada: Opportunities and Challenges

GRADE 9: Canada: Opportunities and Challenges GRADE 9: Canada: Opportunities and Challenges OVERVIEW Grade 9 students will analyze the relationship between Canada s political and legislative processes and their impact on issues pertaining to governance,

More information

Hoboken Public Schools. PLTW Introduction to Computer Science Curriculum

Hoboken Public Schools. PLTW Introduction to Computer Science Curriculum Hoboken Public Schools PLTW Introduction to Computer Science Curriculum Introduction to Computer Science Curriculum HOBOKEN PUBLIC SCHOOLS Course Description Introduction to Computer Science Design (ICS)

More information

INVESTMENT COMMITTEE CHARTER

INVESTMENT COMMITTEE CHARTER INVESTMENT COMMITTEE CHARTER APRIL 2018 HUNTER WATER TABLE OF CONTENTS 1 Committee Name... 3 2 Purpose... 3 3 Objectives... 3 4 Authority... 3 5 Duties and Responsibilities... 3 6 Membership, Appointment

More information

Economic and Social Council

Economic and Social Council UNITED NATIONS E Economic and Social Council Distr. GENERAL 23 July 2008 Original: ENGLISH ECONOMIC COMMISSION FOR EUROPE COMMITTEE ON TRADE Centre for Trade Facilitation and Electronic Business Fourteenth

More information

Towards a Structured Online Consultation Tool

Towards a Structured Online Consultation Tool Towards a Structured Online Consultation Tool Adam Wyner, Katie Atkinson, and Trevor Bench-Capon University of Liverpool, Liverpool, L69 3BX, UK, {azwyner,katie,tbc}@liverpool.ac.uk, http://www.csc.liv.ac.uk/

More information

1 ELECTRONIC COMMUNICATIONS IN CONTRACTUAL TRANSACTIONS 2 DRAFT TABLE OF CONTENTS 3 PART 1 4 GENERAL PROVISIONS

1 ELECTRONIC COMMUNICATIONS IN CONTRACTUAL TRANSACTIONS 2 DRAFT TABLE OF CONTENTS 3 PART 1 4 GENERAL PROVISIONS 1 2 DRAFT TABLE OF CONTENTS 3 PART 1 4 GENERAL PROVISIONS 5 SECTION 101. SHORT TITLE. 6 SECTION 102. DEFINITIONS. 7 SECTION 103. PURPOSES AND CONSTRUCTION 8 SECTION 104. SCOPE. 9 SECTION 105. TRANSACTIONS

More information

DIVISION E--INFORMATION TECHNOLOGY MANAGEMENT REFORM

DIVISION E--INFORMATION TECHNOLOGY MANAGEMENT REFORM DIVISION E--INFORMATION TECHNOLOGY MANAGEMENT REFORM SEC. 5001. SHORT TITLE. This division may be cited as the `Information Technology Management Reform Act of 1995'. SEC. 5002. DEFINITIONS. In this division:

More information

CHAPTER 2 LITERATURE REVIEW

CHAPTER 2 LITERATURE REVIEW 19 CHAPTER 2 LITERATURE REVIEW This chapter presents a review of related works in the area of E- voting system. It also highlights some gaps which are required to be filled up in this respect. Chaum et

More information

FIRST DRAFT VERSION - VISIT

FIRST DRAFT VERSION - VISIT WASH sector coordination is an essential activity in all refugee settings to ensure there is a united and common approach to providing WASH services to the refugee population. Refugee WASH sector coordination

More information

Brexit Transition Support for Local Cymdeithas Llywodraeth Leol Cymru Welsh Local Government Association

Brexit Transition Support for Local   Cymdeithas Llywodraeth Leol Cymru Welsh Local Government Association Cymdeithas Llywodraeth Leol Cymru Welsh Local Government Association Brexit Transition Support for Local Authorities Welsh NHS Confederation Event 22 nd March 2019 @WelshLGA www.wlga.wales Overview WLGA

More information

POLÍCIA JUDICIÁRIA. ASSEMBLEIA DA REPUBLICA T.N. Act no. 73/2009 of 12 August 2009

POLÍCIA JUDICIÁRIA. ASSEMBLEIA DA REPUBLICA T.N. Act no. 73/2009 of 12 August 2009 ASSEMBLEIA DA REPUBLICA T.N. Act no. 73/2009 of 12 August 2009 establishes the conditions and procedures that shall apply to ensure interoperability between information systems of criminal police bodies.

More information

Hoboken Public Schools. Project Lead The Way Curriculum Grade 8

Hoboken Public Schools. Project Lead The Way Curriculum Grade 8 Hoboken Public Schools Project Lead The Way Curriculum Grade 8 Project Lead The Way HOBOKEN PUBLIC SCHOOLS Course Description PLTW Gateway s 9 units empower students to lead their own discovery. The hands-on

More information

2017 Municipal Election Review

2017 Municipal Election Review 2017 Municipal Election Review July 17, 2018 ISC: Unrestricted THIS PAGE LEFT INTENTIONALLY BLANK ISC: Unrestricted Table of Contents Executive Summary... 5 1.0 Background... 7 2.0 Audit Objectives, Scope

More information

Additional Case study UK electoral system

Additional Case study UK electoral system Additional Case study UK electoral system The UK is a parliamentary democracy and hence is reliant on an effective electoral system (Jones and Norton, 2010). General elections are held after Parliament

More information

Servilla: Service Provisioning in Wireless Sensor Networks. Chenyang Lu

Servilla: Service Provisioning in Wireless Sensor Networks. Chenyang Lu Servilla: Provisioning in Wireless Sensor Networks Chenyang Lu Sensor Network Challenges Ø Device heterogeneity Ø Network dynamics q due to mobility and interference Ø Limited resources and energy Signal

More information

TERMS OF REFERENCE. Contracting Authority. 1.0 Beneficiaries. 1.1 Relevant Background SADC EPA

TERMS OF REFERENCE. Contracting Authority. 1.0 Beneficiaries. 1.1 Relevant Background SADC EPA TERMS OF REFERENCE The Design of a Monitoring & Evaluation System for the SADC EPA Member States to track the Operationalization and Impact of the SADC-EU EPA Contracting Authority The Deutsche Gesellschaft

More information

USDL Variant Management. Dr. Daniel Oberle, Senior Researcher, SAP Research Karlsruhe Gunther Stuhec, Standards Architect, SAP AG Walldorf

USDL Variant Management. Dr. Daniel Oberle, Senior Researcher, SAP Research Karlsruhe Gunther Stuhec, Standards Architect, SAP AG Walldorf USDL Variant Management Dr. Daniel Oberle, Senior Researcher, SAP Research Karlsruhe Gunther Stuhec, Standards Architect, SAP AG Walldorf Agenda 1. Problem 2. Solution 1. Grammar : UN/CEFACT Core Component

More information

IBM Cognos Open Mic Cognos Analytics 11 Part nd June, IBM Corporation

IBM Cognos Open Mic Cognos Analytics 11 Part nd June, IBM Corporation IBM Cognos Open Mic Cognos Analytics 11 Part 2 22 nd June, 2016 IBM Cognos Open MIC Team Deepak Giri Presenter Subhash Kothari Technical Panel Member Chakravarthi Mannava Technical Panel Member 2 Agenda

More information

Apply for Hot Works Permit

Apply for Hot Works Permit Apply for Hot Works Permit Details of revisions Level Details Date Initial 1 Creation of initial document for use 20/08/2016 J.O S 2 Review and Amendment 26/02/2018 MW Title: Apply for Hot Works Permit

More information

IC Chapter 15. Ballot Card and Electronic Voting Systems; Additional Standards and Procedures for Approving System Changes

IC Chapter 15. Ballot Card and Electronic Voting Systems; Additional Standards and Procedures for Approving System Changes IC 3-11-15 Chapter 15. Ballot Card and Electronic Voting Systems; Additional Standards and Procedures for Approving System Changes IC 3-11-15-1 Applicability of chapter Sec. 1. Except as otherwise provided,

More information

Functional Requirements for a Secure Electronic Voting System

Functional Requirements for a Secure Electronic Voting System Functional Requirements for a Secure Electronic Voting System Spyros IKONOMOPOULOS 1, Costas LAMBRINOUDAKIS 1, Dimitris GRITZALIS 2, Spyros KOKOLAKIS 1, Kostas VASSILIOU 1 1 Dept. of Information and Communication

More information

AUTOMATED CONTRACT REVIEW

AUTOMATED CONTRACT REVIEW AUTOMATED CONTRACT REVIEW Machine Learning Comes to Corporate Law Session #133 Kingsley Martin KM Standards Amy Harvey & Michael Nogroski Chapman and Cutler SPEAKERS Julian Tsisin Google AUTOMATED CONTRACT

More information

Guidelines for Performance Auditing

Guidelines for Performance Auditing Guidelines for Performance Auditing 2 Preface The Guidelines for Performance Auditing are based on the Auditing Standards for the Office of the Auditor General. The guidelines shall be used as the foundation

More information

U.S. Department of Homeland Security: Improved homeland security management and biometrics through the US-VISIT program

U.S. Department of Homeland Security: Improved homeland security management and biometrics through the US-VISIT program U.S. Department of Homeland Security: Improved homeland security management and biometrics through the US-VISIT program US-Visit In the wake of the attacks of September 11, 2001, the United States federal

More information

The EPO approach to Computer Implemented Inventions (CII) Yannis Skulikaris Director Operations, Information and Communications Technology

The EPO approach to Computer Implemented Inventions (CII) Yannis Skulikaris Director Operations, Information and Communications Technology The EPO approach to Computer Implemented Inventions (CII) Yannis Skulikaris Director Operations, Information and Communications Technology March 2018 Background and context The EPO s approach to CII: fulfills

More information

Case: 1:15-cv SO Doc #: Filed: 08/11/17 1 of 23. PageID #: 3143 EXHIBIT A

Case: 1:15-cv SO Doc #: Filed: 08/11/17 1 of 23. PageID #: 3143 EXHIBIT A Case: 1:15-cv-01046-SO Doc #: 147-1 Filed: 08/11/17 1 of 23. PageID #: 3143 EXHIBIT A Cleveland Police Revised Second-Year Monitoring Plan Case: 1:15-cv-01046-SO Doc #: 147-1 Filed: 08/11/17 2 of 23. PageID

More information

Guidance for Electoral Registration Officers. Part 4 Maintaining the register throughout the year

Guidance for Electoral Registration Officers. Part 4 Maintaining the register throughout the year Guidance for Electoral Registration Officers Part 4 Maintaining the register throughout the year March 2018 Updates to this document Updated September 2013 May 2015 July 2015 November 2015 December 2015

More information

Aim is to simplify and update EU public procurement rules

Aim is to simplify and update EU public procurement rules New EU Directives Richard Heath, Associate New EU Directives One of a package of 3 Directives Directive 2014/23/EU of the European Parliament and of the Council of 26 February 2014 on the award of concession

More information

ABC systems in Europe and beyond - status and recommendations for the way forward

ABC systems in Europe and beyond - status and recommendations for the way forward ABC systems in Europe and beyond - status and recommendations for the way forward Markus Clabian, AIT Austrian Institute of Technology GmbH, Coordinator FastPass Andreas Kriechbaum-Zabini AIT Austrian

More information

Voting Systems Assessment Project

Voting Systems Assessment Project DEAN C. LOGAN Los Angeles County Registrar-Recorder/County Clerk 12400 Imperial Highway Norwalk, CA 90650 Voting Systems Assessment Project Modernizing Los Angeles County Elections July 25, 2012 Productivity

More information

Aristotle s Model of Communication (Devito, 1978)

Aristotle s Model of Communication (Devito, 1978) COMMUNICATION MODELS Models- Definitions In social science research, a model is a tentative description of what a social process, say the communication process or a system might be like. It is a tool of

More information

ForeScout Extended Module for McAfee epolicy Orchestrator

ForeScout Extended Module for McAfee epolicy Orchestrator ForeScout Extended Module for McAfee epolicy Orchestrator Version 3.1 Table of Contents About McAfee epolicy Orchestrator (epo) Integration... 4 Use Cases... 4 Additional McAfee epo Documentation... 4

More information

Committee on Civil Liberties, Justice and Home Affairs WORKING DOCUMENT

Committee on Civil Liberties, Justice and Home Affairs WORKING DOCUMENT EUROPEAN PARLIAMT 2014-2019 Committee on Civil Liberties, Justice and Home Affairs 6.1.2015 WORKING DOCUMT on the Amendment of Regulation (EC) No 562/2006 as regards the use of the Entry/Exit System (EES)

More information

Ø Project Description. Ø Design Criteria. Ø Design Overview. Ø Design Components. Ø Schedule. Ø Testing Criteria. Background Design Implementation

Ø Project Description. Ø Design Criteria. Ø Design Overview. Ø Design Components. Ø Schedule. Ø Testing Criteria. Background Design Implementation Ø Project Description Ø Design Criteria Ø Design Overview Ø Design Components Background Design Implementation Ø Schedule Ø Testing Criteria Ø Asante Solutions, Inc. and RCPD Ø Blind user focused insulin

More information

Mining Toolkit. In-Migration

Mining Toolkit. In-Migration Tool Child Rights and Mining Toolkit Children are the most vulnerable stakeholders regarding mining impacts, including the effects of project-related in-migration. As dependents of migrant mine workers,

More information

DIVISION E INFORMATION TECHNOLOGY MANAGEMENT REFORM

DIVISION E INFORMATION TECHNOLOGY MANAGEMENT REFORM DIVISION E INFORMATION TECHNOLOGY MANAGEMENT REFORM SEC. 5001. SHORT TITLE. This division may be cited as the Information Technology Management Reform Act of 1996. SEC. 5002. DEFINITIONS. In this division:

More information

TRAFFIC NOTE 10. Revision 3. Trials of traffic control devices Guidelines. Date January 2011

TRAFFIC NOTE 10. Revision 3. Trials of traffic control devices Guidelines. Date January 2011 TRAFFIC NOTE 10 Date January 2011 Revision 3 From Authorisation National Planning Unit, Regional Partnerships and Planning Glenn Bunting, Network Manager No. of pages 5 Trials of traffic control devices

More information

For The New Government of Ontario

For The New Government of Ontario For The New Government of Ontario A Ten Point Plan As a private business, charitable foundation and publishing organization, we aim to identify new ideas in the public interest. Issues of settlement have

More information

The Angola National ID Card

The Angola National ID Card The Angola National ID Card Advanced document security for a widely dispersed population 25 by Uwe Ludwig The Republic of Angola in south-central Africa is bordered by Namibia to the South, the Democratic

More information

Overview of the Design Process. Avoid Bad Design, Use UCD Evidence-based Design Hypothesis testing!

Overview of the Design Process. Avoid Bad Design, Use UCD Evidence-based Design Hypothesis testing! Overview of the Design Process Avoid Bad Design, Use UCD Evidence-based Design Hypothesis testing! Good Design (reminder!) Every designer wants to build a highquality interactive system that is admired

More information

Office of the Commissioner of Lobbying of Canada

Office of the Commissioner of Lobbying of Canada Office of the Commissioner of Lobbying of Canada 2013-14 Report on Plans and Priorities The Honourable Tony Clement, PC, MP President of the Treasury Board Table of Contents Message from the Commissioner

More information

World business and the multilateral trading system

World business and the multilateral trading system International Chamber of Commerce The world business organization Policy statement Commission on Trade and Investment Policy World business and the multilateral trading system ICC policy recommendations

More information

Document Approval Process. SDR Forum Policy 001

Document Approval Process. SDR Forum Policy 001 Document Approval Process SDR Forum Policy 001 Revised 27 January, 2009 Scope This policy describes procedures for submission of documents to the SDR Forum for consideration, deliberation, and adoption

More information

Outline. Your Project Proposal an evolving plan. Project Proposal Guidelines (2 files) ECE496 Design Project Preparing Your Project Proposal

Outline. Your Project Proposal an evolving plan. Project Proposal Guidelines (2 files) ECE496 Design Project Preparing Your Project Proposal Outline ECE496 Design Project Preparing Your Project Proposal Dr. Ken Tallman & Khoman Phang Thursday, Sept. 15, 2016 Tonight n Introduction to the proposal (Phang) n Proposal details (Phang) n Writing

More information

Uses and Challenges. Care. Health C. ents in H. ive Age. Normati. Javier Vazquez-Salceda Utrecht University.

Uses and Challenges. Care. Health C. ents in H. ive Age. Normati. Javier Vazquez-Salceda Utrecht University. ve Agealth : Uses and Challenges Javier Vazquez-Salceda Utrecht University it http://www.cs.uu.nl/people/javier lk Invited tal Motivation ealth Motivation o (I) New environment for Health services Need

More information

An Electronic Voting System for a Legislative Assembly

An Electronic Voting System for a Legislative Assembly International Journal of Innovation and Scientific Research ISSN 235-84 Vol. 26 No. 2 Sep. 26, pp. 494-52 25 Innovative Space of Scientific Research Journals http://www.ijisr.issr-journals.org/ An Electronic

More information

Clarifications to this call for applications are presented at the end of this document

Clarifications to this call for applications are presented at the end of this document Clarifications to this call for applications are presented at the end of this document Call for Applications to Conduct Mapping Studies of Trade Unions and Professional Associations as Civil Society Actors

More information

Preamble. THE GOVERNMENT OF THE UNITED STATES OF AMERICA AND THE GOVERNMENT OF THE KINGDOM OF SWEDEN (hereinafter referred to as the Parties ):

Preamble. THE GOVERNMENT OF THE UNITED STATES OF AMERICA AND THE GOVERNMENT OF THE KINGDOM OF SWEDEN (hereinafter referred to as the Parties ): AGREEMENT BETWEEN THE GOVERNMENT OF THE UNITED STATES OF AMERICA AND THE GOVERNMENT OF THE KINGDOM OF SWEDEN ON COOPERATION IN SCIENCE AND TECHNOLOGY FOR HOMELAND SECURITY MATTERS Preamble THE GOVERNMENT

More information

Hoboken Public Schools. Algebra II Honors Curriculum

Hoboken Public Schools. Algebra II Honors Curriculum Hoboken Public Schools Algebra II Honors Curriculum Algebra Two Honors HOBOKEN PUBLIC SCHOOLS Course Description Algebra II Honors continues to build students understanding of the concepts that provide

More information

Secure Electronic Voting: Capabilities and Limitations. Dimitris Gritzalis

Secure Electronic Voting: Capabilities and Limitations. Dimitris Gritzalis Secure Electronic Voting: Capabilities and Limitations Dimitris Gritzalis Secure Electronic Voting: Capabilities and Limitations 14 th European Forum on IT Security Paris, France, 2003 Prof. Dr. Dimitris

More information

Unit 03. Ngo Quy Nham Foreign Trade University

Unit 03. Ngo Quy Nham Foreign Trade University Unit 03 Ngo Quy Nham Foreign Trade University The process by which managers identify organisational problems and try to resolve them. Identifying a problem Identifying decision criteria Allocating weight

More information

Chapter V. Governance and Management Issues of Privatization: Theory & Practice

Chapter V. Governance and Management Issues of Privatization: Theory & Practice Chapter V Governance and Management Issues of Privatization: Theory & Practice DETAILED CONTENTS Contents Page No. Introduction 192 PART-A 193-199 A.1: The Concept of Governance 193 A.2: The Meaning of

More information

Tentative Campus Town Plan Project Timeline

Tentative Campus Town Plan Project Timeline Tentative Campus Town Project Timeline Date Meeting Task Deliverables Prior to Meeting Meeting Objectives Deliverables Post Meeting August 4, August 4, Start of Contract (12 month timer set) Developer

More information

Department of Industrial Engineering: Research Groups

Department of Industrial Engineering: Research Groups Department of Industrial Engineering: Research Groups Engineering Management and Sustainable Systems Operations Management and Supply Chain Management Supply Chain Economics Operations Research Computer

More information

Committee on Development and Intellectual Property (CDIP)

Committee on Development and Intellectual Property (CDIP) E ORIGINAL: ENGLISH DATE: MAY 11, 2012 Committee on Development and Intellectual Property (CDIP) Ninth Session Geneva, May 7 to 11, 2012 SUMMARY BY THE CHAIR 1. The ninth session of the CDIP was held from

More information

Social accountability: What does the evidence really say?

Social accountability: What does the evidence really say? Social accountability: What does the evidence really say? Jonathan Fox School of International Service American University www.jonathan-fox.org fox@american.edu October, 2014 What do evaluations tell us

More information

Introduction to the declination function for gerrymanders

Introduction to the declination function for gerrymanders Introduction to the declination function for gerrymanders Gregory S. Warrington Department of Mathematics & Statistics, University of Vermont, 16 Colchester Ave., Burlington, VT 05401, USA November 4,

More information

Free and Fair elections GUIDANCE DOCUMENT. Commission guidance on the application of Union data protection law in the electoral context

Free and Fair elections GUIDANCE DOCUMENT. Commission guidance on the application of Union data protection law in the electoral context EUROPEAN COMMISSION Brussels, 12.9.2018 COM(2018) 638 final Free and Fair elections GUIDANCE DOCUMENT Commission guidance on the application of Union data protection law in the electoral context A contribution

More information

Software Agents Behaviour.

Software Agents Behaviour. From Human Regulations to einstitutions From Human Regulations to Regulated Software Agents Behaviour. (einstitutions: the KEMLG@UPC and IS@Utrecht view) Javier Vázquez-Salceda May 20, 2005 http://www.lsi.upc.es/~webia/kemlg

More information

PLS 540 Environmental Policy and Management Mark T. Imperial. Topic: The Policy Process

PLS 540 Environmental Policy and Management Mark T. Imperial. Topic: The Policy Process PLS 540 Environmental Policy and Management Mark T. Imperial Topic: The Policy Process Some basic terms and concepts Separation of powers: federal constitution grants each branch of government specific

More information

closer look at Rights & remedies

closer look at Rights & remedies A closer look at Rights & remedies November 2017 V1 www.inforights.im Important This document is part of a series, produced purely for guidance, and does not constitute legal advice or legal analysis.

More information

COURSE PROFILE. Politics of Terrorism POLS 339 Fall Asst. Prof. Özlem Kayhan Pusane. Mehmet Turan Çağlar

COURSE PROFILE. Politics of Terrorism POLS 339 Fall Asst. Prof. Özlem Kayhan Pusane. Mehmet Turan Çağlar COURSE PROFILE Course Name Code Semester Term Theory++Lab (hour/week) Local Credits ECTS Politics of Terrorism POLS 339 Fall 6 3+0+0 3 6 Prerequisites None Course Language Course Type Course Lecturer Course

More information

27 July 2017 Without prejudice TITLE [XX] DIGITAL TRADE

27 July 2017 Without prejudice TITLE [XX] DIGITAL TRADE 27 July 2017 Without prejudice This document is the European Union's (EU) proposal for a legal text on digital trade in the EU- Indonesia FTA. It will be tabled for discussion with Indonesia. The actual

More information

Criminal Justice: Working Together

Criminal Justice: Working Together Report by the Comptroller and Auditor General Lord Chancellor s Department Crown Prosecution Service Home Office Criminal Justice: Working Together Ordered by the House of Commons to be printed 29 November

More information

Prof. Dr. Bernhard Neumärker Summer Term 2016 Albert-Ludwigs-Universität Freiburg. Constitutional Economics. Exam. July 28, 2016

Prof. Dr. Bernhard Neumärker Summer Term 2016 Albert-Ludwigs-Universität Freiburg. Constitutional Economics. Exam. July 28, 2016 Prof. Dr. Bernhard Neumärker Summer Term 2016 Albert-Ludwigs-Universität Freiburg Constitutional Economics Exam July 28, 2016 Please write down your name or matriculation number on every sheet and sign

More information

Secure Electronic Voting

Secure Electronic Voting Secure Electronic Voting Dr. Costas Lambrinoudakis Lecturer Dept. of Information and Communication Systems Engineering University of the Aegean Greece & e-vote Project, Technical Director European Commission,

More information

the third day of January, one thousand nine hundred and ninety-six prescribe personnel strengths for such fiscal year for the Armed

the third day of January, one thousand nine hundred and ninety-six prescribe personnel strengths for such fiscal year for the Armed INFORMATION TECHNOLOGY MANAGEMENT REFORM ACT (Now the Clinger/Cohen Act) s.1124 One Hundred Fourth Congress of the United States of America AT THE SECOND SESSION Begun and held at the City of Washington

More information

Alex LeBlanc, New Brunswick Multicultural Council P2P Toronoto, November 17, 2017 NouLAB

Alex LeBlanc, New Brunswick Multicultural Council P2P Toronoto, November 17, 2017 NouLAB Turning the Tide on Demographic and Labour Market Decline: How Social Innovation Labs are Introducing User-Centered Design, Systems-Thinking and Experimentation to Improve Settlement, Integration and Retention

More information

Migration & Gender: Vocational and Educational counseling - MOVE ON Kick-off meeting

Migration & Gender: Vocational and Educational counseling - MOVE ON Kick-off meeting Migration & Gender: Vocational and Educational counseling - MOVE ON Kick-off meeting Mar Camarasa i Casals, Laura Sales Gutiérrez Surt Foundation, Barcelona London, 19 th November 2015 The Programme is

More information

OASIS ELECTION AND VOTER SERVICES TECHNICAL COMMITTEE. ELECTION MARK-UP LANGUAGE (EML): e-voting PROCESS AND DATA REQUIREMENTS

OASIS ELECTION AND VOTER SERVICES TECHNICAL COMMITTEE. ELECTION MARK-UP LANGUAGE (EML): e-voting PROCESS AND DATA REQUIREMENTS OASIS ELECTION AND VOTER SERVICES TECHNICAL COMMITTEE ELECTION MARK-UP LANGUAGE (EML): e-voting PROCESS AND DATA REQUIREMENTS Version 2 5 September 2002 Page No 1 Document Control Abstract This document

More information

City Planning & Environmental Services. 2 September 2010

City Planning & Environmental Services. 2 September 2010 City Planning & Environmental Services 2 September 2010 Mr. A Gardner Clerk of the Committee Primary Production Select Committee Parliament House WELLINGTON Dear Sir SUBMISSION TO THE FOOD BILL 1.0 INTRODUCTION

More information

ROSE-HULMAN INSTITUTE OF TECHNOLOGY POLICY REGARDING INTELLECTUAL PROPERTY

ROSE-HULMAN INSTITUTE OF TECHNOLOGY POLICY REGARDING INTELLECTUAL PROPERTY ROSE-HULMAN INSTITUTE OF TECHNOLOGY POLICY REGARDING INTELLECTUAL PROPERTY (Adopted by the Board of Managers on February 24, 1989 now referred to as Board of Trustees) The primary mission of Rose-Hulman

More information

Terms of Reference 1. INTRODUCTION

Terms of Reference 1. INTRODUCTION Terms of Reference Description of the assignment (Title of consultancy): National Consultant to assess the Regional Technical Office (RTO) operating under the mandate of the Union of Municipalities of

More information

The UK s Migration Statistics Improvement Programme - exploiting administrative sources to improve migration estimates

The UK s Migration Statistics Improvement Programme - exploiting administrative sources to improve migration estimates Distr.: General 10 October 2012 Original: English Working paper 12 Economic Commission for Europe Conference of European Statisticians Group of Experts on Migration Statistics Work Session on Migration

More information

ALTERNATIVES TO ADJUDICATION. Toby Randle. 9 May 2005 THE SAVOY HOTEL, LONDON

ALTERNATIVES TO ADJUDICATION. Toby Randle. 9 May 2005 THE SAVOY HOTEL, LONDON ALTERNATIVES TO ADJUDICATION 11 TH ADJUDICATION UPDATE SEMINAR Toby Randle 9 May 2005 THE SAVOY HOTEL, LONDON Here I am, at the 11 th Fenwick Elliott adjudication seminar, in a room full of people closely

More information

FOOD SECURITY OUTCOME MONITORING : SYRIAN REFUGEES IN JORDAN

FOOD SECURITY OUTCOME MONITORING : SYRIAN REFUGEES IN JORDAN FOOD SECURITY OUTCOME MONITORING : SYRIAN REFUGEES IN JORDAN TERMS OF REFERENCE- DRAFT NOVEMBER 2015 SUMMARY As the Syrian refugee crisis enters its fifth year, rates of external assistance have begun

More information

City of Toronto Election Services Internet Voting for Persons with Disabilities Demonstration Script December 2013

City of Toronto Election Services Internet Voting for Persons with Disabilities Demonstration Script December 2013 City of Toronto Election Services Internet Voting for Persons with Disabilities Demonstration Script December 2013 Demonstration Time: Scheduled Breaks: Demonstration Format: 9:00 AM 4:00 PM 10:15 AM 10:30

More information

A Iterative Analysis to Improve Key Properties of Critical Human-Intensive Processes: An Election Security Example

A Iterative Analysis to Improve Key Properties of Critical Human-Intensive Processes: An Election Security Example A Iterative Analysis to Improve Key Properties of Critical Human-Intensive Processes: An Election Security Example Leon J. Osterweil, University of Massachusetts Amherst Matt Bishop, University of California

More information

ISLAMABAD, MONDAY, SEPTEMBER 27, 2004 PART I. Acts, Ordinances, President's Orders and Regulations SENATE SECRETARIAT

ISLAMABAD, MONDAY, SEPTEMBER 27, 2004 PART I. Acts, Ordinances, President's Orders and Regulations SENATE SECRETARIAT REGISTERED No. M - 302 L.-7646 EXTRAORDINARY PUBLISHED BY AUTHORITY ISLAMABAD, MONDAY, SEPTEMBER 27, 2004 PART I Acts, Ordinances, President's Orders and Regulations SENATE SECRETARIAT Islamabad, the 25th

More information

BA 20/20 Becoming a Benchmark Company for the Business Analyst Practice. BBC 2014 Building Business Capability

BA 20/20 Becoming a Benchmark Company for the Business Analyst Practice. BBC 2014 Building Business Capability BA 20/20 Becoming a Benchmark Company for the Business Analyst Practice BBC 2014 Building Business Capability @BBCapability #BBCCon @HansEckman HansEckman.com Ground Rules This session is for you, so please

More information

ON-LINE CONTEST (SWEEPSTAKES) Strongbow Big Apple Sweeps (the Contest )

ON-LINE CONTEST (SWEEPSTAKES) Strongbow Big Apple Sweeps (the Contest ) July/2014 Page 1 of 5 OFFICIAL CONTEST RULES ON-LINE CONTEST (SWEEPSTAKES) Strongbow Big Apple Sweeps (the Contest ) 1. ELIGIBILITY: To be eligible to enter, you must be: (i) a resident of British Columbia;

More information

Federated Decision Making

Federated Decision Making Federated Decision Making By Roy Zuniga July 2011 Duvall, WA The people feel disconnected from the decision of government made so far away that impact their daily lives. The question comes up, then, of

More information

LESAT Facilitator s! Workshop!

LESAT Facilitator s! Workshop! LESAT Facilitator s Workshop LESAT Facilitation March 28, 2002 Presented by: Dr. Joe Mize Massachusetts Institute of Technology Research Sponsored By LAI Lean Aerospace Initiative Lean Transformation Requires

More information

Office of the Commissioner of Lobbying of Canada. Report on Plans and Priorities. The Honourable Tony Clement, PC, MP President of the Treasury Board

Office of the Commissioner of Lobbying of Canada. Report on Plans and Priorities. The Honourable Tony Clement, PC, MP President of the Treasury Board Office of the Commissioner of Lobbying of Canada 2011 12 Report on Plans and Priorities The Honourable Tony Clement, PC, MP President of the Treasury Board Table of Contents Message from the Commissioner

More information

PNC Inspections: National overview report

PNC Inspections: National overview report PNC Inspections: National overview report 4 August 2010 1 Contents Introduction Background National themes Conclusion Annex A Leadership and strategic direction Partnerships Preventing system abuse Performance

More information

Item 8 of the Provisional Agenda SEVENTH SESSION OF THE GOVERNING BODY. Kigali, Rwanda, 30 October 3 November 2017

Item 8 of the Provisional Agenda SEVENTH SESSION OF THE GOVERNING BODY. Kigali, Rwanda, 30 October 3 November 2017 September 2017 IT/GB-7/17/31 Item 8 of the Provisional Agenda SEVENTH SESSION OF THE GOVERNING BODY Kigali, Rwanda, 30 October 3 November 2017 CO-CHAIRS PROPOSAL FROM THE OUTCOMES OF THE MEETINGS OF THE

More information

MARTIN LUTHER UNIVERSITY HALLE-WITTENBERG. Senate

MARTIN LUTHER UNIVERSITY HALLE-WITTENBERG. Senate OFFICIAL JOURNAL MARTIN LUTHER UNIVERSITY HALLE-WITTENBERG 19 th Year, No. 5, dated 2 June 2009, p. 14 Senate Statute establishing the guidelines for safeguarding good academic practice and the treatment

More information