The Rt Hon. Lord Willetts FRS is the President of the Resolution Foundation. He served as the Member of Parliament for Havant (1992-2015), as Minister for Universities and Science (2010-2014) and previously worked at HM Treasury and the No. 10 Policy Unit.
Lord Willetts is a visiting Professor at King’s College London, a Board member of UK
The Rt Hon. Lord Willetts FRS is the President of the Resolution Foundation. He served as the Member of Parliament for Havant (1992-2015), as Minister for Universities and Science (2010-2014) and previously worked at HM Treasury and the No. 10 Policy Unit.
Lord Willetts is a visiting Professor at King’s College London, a Board member of UK Research and Innovation (UKRI), a Board member of Surrey Satellites and of the Biotech Growth Trust. He is the Chair of the Sanger Institute and the Chair of Foundation for Science and Technology. He is an Honorary Fellow of Nuffield College, an Honorary Fellow of the Royal Society and the Chancellor of the University of Leicester.
Lord Willetts has written widely on economic and social policy. His book “A University Education” is published by Oxford University Press. A second edition of his book on the Boomers and the young generation “The Pinch” will be published in November.
David Hand is a co-proposer of the Validate AI Conference. He is Senior Research Investigator and Emeritus Professor of Mathematics at Imperial College, London, where he previously held the Chair of Statistics. He serves on the Board of the UK Statistics Authority and the European Statistical Advisory Committee. He is a former president
David Hand is a co-proposer of the Validate AI Conference. He is Senior Research Investigator and Emeritus Professor of Mathematics at Imperial College, London, where he previously held the Chair of Statistics. He serves on the Board of the UK Statistics Authority and the European Statistical Advisory Committee. He is a former president of the Royal Statistical Society and has received many awards for his research, including the Guy Medal of the Royal Statistical Society, the Box Medal from the European Network for Business and Industrial Statistics, and the Research Award of the International Federation of Classification Societies. His 29 books include Principles of Data Mining, Measurement Theory and Practice, The Improbability Principle, Information Generation, and Intelligent Data Analysis.
Marta Kwiatkowska is a co-proposer of the Validate AI Conference. She is Professor of Computing Systems and Fellow of Trinity College, University of Oxford. Prior to this she was Professor in the School of Computer Science at the University of Birmingham, Lecturer at the University of Leicester and Assistant Professor at the Jagiellonian
Marta Kwiatkowska is a co-proposer of the Validate AI Conference. She is Professor of Computing Systems and Fellow of Trinity College, University of Oxford. Prior to this she was Professor in the School of Computer Science at the University of Birmingham, Lecturer at the University of Leicester and Assistant Professor at the Jagiellonian University in Cracow, Poland. Kwiatkowska has made fundamental contributions to the theory and practice of model checking for probabilistic systems, focusing on automated techniques for verification and synthesis from quantitative specifications. More recently, she has been working on safety and robustness verification for neural networks with provable guarantees. She led the development of the PRISM model checker, the leading software tool in the area and winner of the HVC Award 2016. Probabilistic model checking has been adopted in diverse fields, including distributed computing, wireless networks, security, robotics, healthcare, systems biology, DNA computing and nanotechnology, with genuine flaws found and corrected in real-world protocols. Kwiatkowska is the first female winner of the Royal Society Milner Award and was awarded an honorary doctorate from KTH Royal Institute of Technology in Stockholm. She won two ERC Advanced Grants, VERIWARE and FUN2MODEL, the latter focusing on developing probabilistic verification methods for deep learning, and is a coinvestigator of the EPSRC Programme Grant on Mobile Autonomy. Kwiatkowska is a Fellow of the Royal Society, Fellow of ACM, Member of Academia Europea and Fellow of the BCS.
Michael Wooldridge is a Professor of Computer Science and Head of Department of Computer Science at the University of Oxford, and programme director for AI at the Alan Turing Institute. He has been an AI researcher for more than 25 years, and has published more than 350 scientific articles on the subject. In particular, he is a of the in
Michael Wooldridge is a Professor of Computer Science and Head of Department of Computer Science at the University of Oxford, and programme director for AI at the Alan Turing Institute. He has been an AI researcher for more than 25 years, and has published more than 350 scientific articles on the subject. In particular, he is a of the international leaders in the field of multi-agent systems. He is a Fellow of the Association for Computing Machinery (ACM), the Association for the Advancement of AI (AAAI), and the European Association for AI (EurAI). From 2014-16, he was President of the European Association for AI, and from 2015-17 he was President of the International Joint Conference on AI (IJCAI).
Dr Faisal is Reader in Neurotechnology jointly at the Dept. of Bioengineering and the Dept. of Computing at Imperial College London, where he leads the Brain & Behaviour Lab. Aldo is also Director of the Behaviour Analytics Lab at the Data Science Institute. He is also Associate Investigator at the MRC London Institute of Medical Sciences
Dr Faisal is Reader in Neurotechnology jointly at the Dept. of Bioengineering and the Dept. of Computing at Imperial College London, where he leads the Brain & Behaviour Lab. Aldo is also Director of the Behaviour Analytics Lab at the Data Science Institute. He is also Associate Investigator at the MRC London Institute of Medical Sciences and is affiliated faculty at the Gatsby Computational Neuroscience Unit (University College London). Aldo serves as an Associate editor for Nature Scientific Data and PLOS Computational Biology and has acted as conference chair, area chair, program chair in key conferences in the field (e.g. Neurotechnix, KDD, NIPS, IEEE BSN), in 2016 he was elected into the Global Futures Council of the World Economic Forum.
Aldo received a number of awards and distinctions, including being scholar of the German National Merit Foundation (Studienstiftung des Deutsche Volkes; Undergraduate PhD), a Fellow of the Böhringer-Ingelheim Foundation for Basic Biomedical Research, elected as a Junior Research Fellow at the University of Cambridge (Wolfson College), and research prizes such as the Toyota Mobility Foundation award in 2018 ($50,000).
Aldo read Computer Science and Physics in Germany, where he wrote his Diplomarbeit (M.Sc. thesis) in non-linear dynamical systems and neural networks (with Helge Ritter). He moved on to study Biology at Cambridge University (Emmanuel College) and wrote his M.Phil. thesis on the electrophysiological and behavioural study of a complex motor behaviour in freely moving insects with Tom Matheson in the group of Malcolm Burrow FRS. For his Ph.D. he joined Simon Laughlin FRS group at the Zoology Department in Cambridge investigating the biophysical sources of neuronal variability. He was elected a Junior Research Fellow at Cambridge University (Wolfson College) and joined the Computational & Biological Learning Group (Engineering Department) to work with Daniel Wolpert FRS on human sensorimotor control. Between and after his studies he gained insights into strategic mangement consulting with McKinsey & Co. and as a "quant" with the investment bank Credit Suisse. In winter 2009 Aldo setup the Brain & Behaviour Lab at Imperial College to pursue a research program that aims at understanding the brain with principles from engineering which often immediately translates into direct technological applications for patients and society.
Frankie is responsible for the capability of ONS, ensuring that everybody has the technology and skills to enable them to do their job effectively. She is leading the digital and workforce transformation to allow us to collect, securely store and analyse data from new sources. Frankie champions openness and data sharing for the public go
Frankie is responsible for the capability of ONS, ensuring that everybody has the technology and skills to enable them to do their job effectively. She is leading the digital and workforce transformation to allow us to collect, securely store and analyse data from new sources. Frankie champions openness and data sharing for the public good in ONS and beyond.
Frankie joined the ONS in January 2010 as Head of IT Project Delivery. Since then, she has held several senior roles in ONS, including leading the development strategy for the UK National Accounts. Most recently, Frankie has led the transformation of ONS economic and population and public policy statistics, taking a leading role on preparations for Census 2021.
Shakeel is a co-proposer and chair of the committee of the Validate AI Conference. He has been a great advocate of Artificial Intelligence supporting capability building in HMRC as well as sharing his expertise across government departments and tax administrations globally over the last decade. Prior to this he worked in Financial Servic
Shakeel is a co-proposer and chair of the committee of the Validate AI Conference. He has been a great advocate of Artificial Intelligence supporting capability building in HMRC as well as sharing his expertise across government departments and tax administrations globally over the last decade. Prior to this he worked in Financial Services leading some major supervised and unsupervised machine learning initiatives for 10 years. In the last five years he has worked closely with academics with extensive tacit industry knowledge to develop a novel Data Science Masterclass program. He graduated with a Masters in Operational Research at Strathclyde Business School which has greatly aided his ability over his career to deliver in the AI field combining maths, programming and problem structuring methods. His extensive experience across private, public and academic sectors has also enabled him to benefit from the notion of the triple helix model.
Dr Jasmine Grimsley is a Senior Data Scientist at the Data Science Campus at the Office for National Statistics. She links Academia and Government on research surrounding Data Science for the Public Good both nationally and internationally. Her current interests include understanding the ethics of AI, and using data mining, machine learn
Dr Jasmine Grimsley is a Senior Data Scientist at the Data Science Campus at the Office for National Statistics. She links Academia and Government on research surrounding Data Science for the Public Good both nationally and internationally. Her current interests include understanding the ethics of AI, and using data mining, machine learning, and natural language processing to solve real world problems.
She previously headed a government funded neuroethology research laboratory within a medical school in the USA, where she still holds a position as an Adjunct Professor (NEOMED). She completed a PhD at the MRC Institute for Hearing Research in Biomedical Science where she developed a love for using large multidimensional data sets to understand systems. Her BSc was in Psychology and Neuropsychology from the University of Wales Bangor.
Jonathan Crook is Director of the Credit Research Centre at the University of Edinburgh as well as Deputy Dean and Director of Research at the University of Edinburgh Business School. He has researched into credit scoring and the demand and supply of credit and credit constraints since the late 1980s. He has published over 55 refereed jo
Jonathan Crook is Director of the Credit Research Centre at the University of Edinburgh as well as Deputy Dean and Director of Research at the University of Edinburgh Business School. He has researched into credit scoring and the demand and supply of credit and credit constraints since the late 1980s. He has published over 55 refereed journal articles, mostly in the areas of credit scoring and credit availability in journals such as the European Journal of Operational Research, Journal of the Operational Research Society, Journal of Banking and Finance, Journal of Business Finance and Accounting and Economics Letters. He has co-authored or co-edited five books including Credit Scoring and its Applications, 2017 (with Lyn Thomas and David Edelman). He is particularly interested in the incorporation of macroeconomic factors into credit risk modelling, stress testing, survival models, dynamic models, the demand and supply of credit and the use of new forms of data. He has received research grants from the EPSRC, ESRC, Fulbright and other sources. He is a Fellow of the Financial Institutions Center at the Wharton School, University of Pennsylvania and an External Research Fellow of the Centre for Finance, Credit and Macroeconomics at the University of Nottingham. He has acted as a credit scoring consultant to a number of banks. He is also the joint Editor of the Journal of the Operational Research Society, a Fellow of the Royal Society of Edinburgh and a Fellow of the Academy of Social Sciences.
As head of the Data Science at Capital One UK I lead an 20-strong cross-functional team that is changing the way we do business. We use the latest distributed computing technologies and operate across many billions of customer transactions. The models and data products that my team build unlock big opportunities for the business and help
As head of the Data Science at Capital One UK I lead an 20-strong cross-functional team that is changing the way we do business. We use the latest distributed computing technologies and operate across many billions of customer transactions. The models and data products that my team build unlock big opportunities for the business and help UK consumers save money and reduce friction in their financial lives.
Stan Boland is a co-founder and CEO of Five, a major start-up developing autonomous vehicle technology based in the UK founded in 2015. Five has so far built and demonstrated a self-driving capability for London streets and a number of high value offline tools in support of that effort. He was previously co-founder and CEO of two of the
Stan Boland is a co-founder and CEO of Five, a major start-up developing autonomous vehicle technology based in the UK founded in 2015. Five has so far built and demonstrated a self-driving capability for London streets and a number of high value offline tools in support of that effort. He was previously co-founder and CEO of two of the UK’s most successful venture-backed communications silicon and software companies, Element 14 Inc. and Icera Inc., bought by Broadcom and NVIDIA respectively, for an aggregate value of over $1 billion. Much earlier he was CEO of Acorn, a pioneer computer and technology firm and a Board member of its one-time associate, ARM. He is a graduate in physics from the University of Cambridge.
Dr Iain Whiteside is the Principal Scientist and Lead on Assurance at Five. Iain was formerly an Assurance Researcher in unmanned aerial vehicles (UAVs) at NASA Ames facility, California. Whilst at NASA he was the grant holder for NASA’s participation in the DARPA ‘Assured Autonomy’ program. He earned his PhD in formal software verification at the University of Edinburgh.
Michael Bronstein joined the Department of Computing as Professor in 2018. He has served as a professor at USI Lugano, Switzerland since 2010 and held visiting positions at Stanford, Harvard, MIT, TUM, and Tel Aviv University. Michael received his PhD with distinction from the Technion (Israel Institute of Technology) in 2007. His main e
Michael Bronstein joined the Department of Computing as Professor in 2018. He has served as a professor at USI Lugano, Switzerland since 2010 and held visiting positions at Stanford, Harvard, MIT, TUM, and Tel Aviv University. Michael received his PhD with distinction from the Technion (Israel Institute of Technology) in 2007. His main expertise is in theoretical and computational geometric methods for data analysis, and his research encompasses a broad spectrum of applications ranging from machine learning, computer vision, and pattern recognition to geometry processing, computer graphics, and imaging.
Michael has authored over 150 papers, the book Numerical geometry of non-rigid shapes (Springer 2008), and holds over 30 granted patents. He was awarded four ERC grants, two Google Faculty Research awards, Amazon ML Research award, Facebook Computational Social Science award, Dalle Molle prize, and the Royal Society Wolfson Merit award. During 2017-2018 he was a fellow at the Radcliffe Institute for Advanced Study at Harvard University and since 2017, he is a Rudolf Diesel fellow at TU Munich. He was invited as a Young Scientist to the World Economic Forum, an honor bestowed on forty world’s leading scientists under the age of forty. Michael is a Fellow of IEEE and IAPR, alumnus of the Technion Excellence Program and the Academy of Achievement, ACM Distinguished Speaker, and a member of the Young Academy of Europe. In addition to academic work, Michael's industrial experience includes technological leadership in multiple startup companies, including Novafora, Videocites, and Invision (acquired by Intel in 2012), and Fabula AI (acquired by Twitter in 2019). Following the acquisition of Fabula, he joined Twitter as Head of Graph Learning Research. He previously served as Principal Engineer at Intel Perceptual Computing (2012-2019) and was one of the key developers of the Intel RealSense 3D camera technology.
Pushmeet Kohli is the Head of research for the "AI for science" and "Robust & Reliable AI" groups at DeepMind. His research revolves around Intelligent Systems and Computational Sciences. His current research interests include Safe and Verified AI, Probabilistic Programming, Interpretable and Verifiable Knowledge Representations of Deep
Pushmeet Kohli is the Head of research for the "AI for science" and "Robust & Reliable AI" groups at DeepMind. His research revolves around Intelligent Systems and Computational Sciences. His current research interests include Safe and Verified AI, Probabilistic Programming, Interpretable and Verifiable Knowledge Representations of Deep Models. In terms of application domains, he is interested in goal-directed conversation agents, machine learning systems for healthcare, and 3D reconstruction and tracking for augmented and virtual reality. Pushmeet’s papers has appeared in conferences in the field of machine learning, computer vision, game theory and human computer interaction and have won awards in CVPR, WWW, CHI, ECCV, ISMAR, TVX and ICVGIP. His research has also been the subject of a number of articles in popular media outlets such as Forbes, Wired, BBC, New Scientist and MIT Technology Review. Pushmeet is on the editorial board or PAMI, JMLR, IJCV and CVIU.
Giles Herdale is co-chair of the Independent Digital Ethics Panel for Policing working with leading experts from academia, law, ethics, civil society and digital investigation to advise on effective and ethical digital investigation. He has been involved in digital investigation for many years, working in policing at the NPIA, College of
Giles Herdale is co-chair of the Independent Digital Ethics Panel for Policing working with leading experts from academia, law, ethics, civil society and digital investigation to advise on effective and ethical digital investigation. He has been involved in digital investigation for many years, working in policing at the NPIA, College of Policing and NPCC, where he set up and ran the DII programme. He has also had experience in the private sector providing digital forensics and data analytics services to law enforcement and the criminal justice system. He is now a consultant and policy advisor working on the challenge of the changing face of crime, investigation and intelligence in the digital age.
Stephanie Hare is an independent researcher and broadcaster focused on technology, politics and history. Previously she worked as a Principal Director at Accenture Research, a strategist at Palantir, a Senior Analyst at Oxford Analytica, the Alistair Horne Visiting Fellow at St Antony's College, Oxford, and a consultant at Accenture. She
Stephanie Hare is an independent researcher and broadcaster focused on technology, politics and history. Previously she worked as a Principal Director at Accenture Research, a strategist at Palantir, a Senior Analyst at Oxford Analytica, the Alistair Horne Visiting Fellow at St Antony's College, Oxford, and a consultant at Accenture. She holds a PhD and MSc from the London School of Economics and a BA in Liberal Arts and Sciences (French) from the University of Illinois at Urbana-Champaign.
Zeynep is a Senior Research Associate at UCL Computer Science also leading UCL's Digital Ethics Forum - an EPSRC IAA funded platform developing cross-disciplinary responses to major societal issues caused by the ongoing 'digital revolution'. She is the Founder and Chair of the international Data for Policy conferences (dataforpolicy.org)
Zeynep is a Senior Research Associate at UCL Computer Science also leading UCL's Digital Ethics Forum - an EPSRC IAA funded platform developing cross-disciplinary responses to major societal issues caused by the ongoing 'digital revolution'. She is the Founder and Chair of the international Data for Policy conferences (dataforpolicy.org), the Editor-in-Chief for Data and Policy (cambridge.org/dap) - an open-access peer-reviewed publication venue developed in collaboration with Cambridge University Press, and the Principal Investigator of the GovTech Lab (govtechlab.org).
Carly Kind is the Director of the Ada Lovelace Institute, an independent research body and think tank with a mission to ensure data and AI work for people and society. A human rights lawyer and leading authority on the intersection of technology policy and human rights, Carly has advised industry, government and non-profit organisations
Carly Kind is the Director of the Ada Lovelace Institute, an independent research body and think tank with a mission to ensure data and AI work for people and society. A human rights lawyer and leading authority on the intersection of technology policy and human rights, Carly has advised industry, government and non-profit organisations on digital rights, privacy and data protection, and corporate accountability. She has worked with the European Commission, the Council of Europe, numerous UN bodies and a range of civil society organisations. She was formerly Legal Director of Privacy International, an NGO dedicated to promoting data rights and governance.
Martin is Chief Scientist and CEO of Evolution AI, and a specialist in machine reading technologies. He is the Chair of the Royal Statistical Society Data Science Section, the professional body for data science in the UK. He also runs the largest machine learning community in Europe, Machine Learning London. Martin’s work has been covere
Martin is Chief Scientist and CEO of Evolution AI, and a specialist in machine reading technologies. He is the Chair of the Royal Statistical Society Data Science Section, the professional body for data science in the UK. He also runs the largest machine learning community in Europe, Machine Learning London. Martin’s work has been covered by publications such as The Economist, Quartz, Business Insider and Tech Crunch.
Tom Smith is Managing Director at the Data Science Campus, joining the Office for National Statistics (ONS) in 2017. He was co-founder and, prior to joining ONS, chief executive of Oxford Consultants for Social Inclusion (OCSI), a research and data ‘spin-out’ company from the University of Oxford.
Tom has more than 20 years’ experience us
Tom Smith is Managing Director at the Data Science Campus, joining the Office for National Statistics (ONS) in 2017. He was co-founder and, prior to joining ONS, chief executive of Oxford Consultants for Social Inclusion (OCSI), a research and data ‘spin-out’ company from the University of Oxford.
Tom has more than 20 years’ experience using data and analysis to improve public services. Working at the intersection of government, academia and industry, he has led data & research projects with hundreds of local and national public and community sector organisations, including the government’s English Indices of Deprivation. His primary research interests are in using data science to improve public services, machine learning, and assessing non-traditional data sources to improve our understanding of society and the economy.
A life-long data addict, Tom has a PhD in computational neuroscience, evolving neural networks for robot control (Sussex, 2002), an MSc in knowledge-based systems (Sussex, 1997), and MA in theoretical physics (Cambridge, 1994). He is vice-chair of the Royal Statistical Society Official Statistics section, and previously chair of the Environment Agency Data Advisory Group, and a member of the Open Data User Group ministerial advisory group to Cabinet Office. He has also acted as an external advisor on opening-up, sharing and using data for multiple government departments.
Lord Willetts began the conference by stressing the importance of the topic, suggesting that AI was now too deeply embedded in society for a third AI winter to occur. He drew attention to the carbon footprint of computer systems and the key challenge of how AI systems interact with humans. Ethical matters of data science, machine learning, and AI are attracting growing attention, with several bodies focusing on aspects of them having been established recently. The explainability of AI decisions was a key aspect of their acceptability, but we need to recognise that “artificial intelligence” is not the same as “natural intelligence”.
David Hand presented eight dimensions of invalidity in AI systems, giving examples of each. The dimensions were: (1) Ignorance of the mechanisms of AI systems; (2) Ignorance of the limitations of AI systems; (3) Defining and ensuring the limits of behaviour; (4) Adequacy of data; (5) Robustness to perturbations of data; (6) Requiring assurance that AI systems do what they are supposed to do in familiar situations; (7) How AI systems behave with insufficiently specified problems; (8) How people use, or work with systems. He suggested that AI systems were confronting us with a new kind of principal agent problem, and that the law of unintended consequences was likely to manifest often. He consolidated the validation questions as (a) what properties must an AI system have for us to trust its decisions? (b) how can we ensure it has these properties? He also illustrated how the choice of evaluation method and performance measure can lead to dramatically different assessments of statistical models, highlighting the need for care when determining how to validate an AI system.
Marta Kwiatkowska observed that the data aspects discussed by Hand and the code aspects she would discuss were the two foundational aspects of AI. While the ambition was strong AI, currently at best we had only weak AI. She illustrated the challenges of validation with some less successful projects: IBM cancelling its Watson oncology project and Apple face-identification being defeated by a 3D printed face mask. She stressed the key role of trust in AI systems, drawing attention to the complex scenarios in which such systems would function, as well as the challenges of safety critical systems. Provable guarantees of performance were needed, distinguishing between verification, being proof the system satisfies its specification, and validation, being confirmation that it was fit for purpose. It is critically important to involve domain experts, since machine learning is very different from conventional programming. The former involves black box programming by pattern matching from examples, and it is important to ensure that systems are developed for users, not the developers themselves. Development must take place within the context of trust, ethics, morality, and social norms.
Michael Wooldridge focused on multiple interacting systems, which need to cooperate, coordinate, and negotiate with each other. He gave the example of financial trading systems, referring to the 2010 Flash Crash as an example of how hidden correlations between systems which behave in the same way can cause dramatic unexpected consequences and unstable behaviour. The ultimate goal in AI systems was not a system which has to be told what to do but one that works to help you. This means that systems need to know your preferences and values. This leads to the preference elicitation problem: how to code up our requirements for ethical and trustworthy behaviour in AI. He noted that unstable equilibria were a particular challenge for multi-agent systems, and described the two strategies for understanding and tackling the issues: treat the problems (e.g. the Flash Crash) as a bug vs use agent-based modelling and simulation.
Aldo Faisal discussed diagnostic systems which mimic clinicians’ perceptual ability to assess a situation – a very successful illustration of weak AI. He stressed the value of reinforcement learning, and of trust and explainability. This is partly a problem of human cognition: what is it we want to be explained and how? Do we want to interpret the AI or explain one specific decision made by an AI? One particular challenge arose from the regulators’ requirement that systems should be fixed and not change and adapt (in unpredictable ways) as they autonomously learn.
Frankie Kay discussed the increasing use and challenges of combinations of data from multiple sources in government and the potential of efficiency and improved opportunities for applying AI. For example, in policing, we now have data from CCTV, biometrics and, increasingly, the Internet-of-Things. Dealing with this varied data requires that we ensure the quality of the data, that it is unbiased and fit for the purpose for which it is intended. It is important that industry and government using AI have staff with the technical skills to implement these systems correctly.
Shakeel Khan described his experience of AI capability building and in particular development (identify/adopt/innovate), knowledge advancement (explicit/tacit), and cross-sector collaboration (the triple helix framework). He stressed the importance of robustness across the entire life cycle and the need for proper validation management policies, drawing attention to The Predictive Analytics Handbook used at HMRC for development of predictive models. A particular challenge he identified was whether a system is fit for purpose as the population it is applied to changes; this is a common problem for systems that model human behaviour, such as credit scoring models. He suggested the use of sensitivity analysis or stress testing as a means to check for robustness to a changing population. Other issues identified were problems with future data loads, external economic or political factors, and the difficulties of predictive data not being available in the future.
Jasmine Grimsley looked at the use of AI in government, stressing AI project ethics in law (e.g. the GDPR), internal policies (e.g. ONS web-scraping policies), and advisory guidelines (e.g. EU ethical AI guidelines). Consideration had to be given to personal as well as social ethics.
Jonathan Crook described some of the challenges of validating AI systems in consumer finance, with focus on predicting default. Topics he discussed included sample distortion, unbalanced data, interpretability, and indirect bias in different groups leading to challenges to fairness.
Dan Kellett described the drift from relatively simple models in the consumer finance industry, such as logistic regression trees, to more sophisticated and less transparent models, such as gradient boosted methods. Validation questions he posed included: how does the model behave when presented with data values outside its previous range, is the model degrading over time, how well does the model perform on different products, is the model appropriate for a new population, and is the environment changing (e.g. economic changes)? Sensitivity analysis is useful for this purpose - testing how the model performs when there are deviations in values of variables. He also pointed out the need to be able to answer the questions of how well we understand the data, how well we understand the model, and how will the model interact with downstream processes. He emphasized the need to properly understand the business (application) problem the model is intended to address and to ensure the appropriate use of models (i.e. a model built for one purpose may not be suitable for another).
Stan Boland discussed validation issues with self-driving cars. He suggested that self-driving cars are close to implementation but that currently their failure rates are too high (order of 10-3 failures/decision) relative to failure rates among human drivers (estimated as an order of 10-7 failures/decision). This is due to a very long tail of perception problems. Therefore, there is a great deal of room for improvement. Noting that it was not possible to model or simulate all possible scenarios an autonomous vehicle might encounter, he drew attention to the need to model the relationship between physical invariants (such as rain) and the error rate of perception systems, in an attempt to alleviate brittleness risks. He proposed a framework for this based on simulating perception systems through generating all salient aspects, adding noise to images and deliberately simulating adversarial examples to improve robustness, generating new directed tests, measuring performance, and establishing metrics. Simulation is a large part of the development and testing process for self-driving cars. Skills and experience of programmers from the gaming industry are employed to ensure the development of realistic driving simulations. For validation, standard machine learning evaluation can be supplemented by domain knowledge to ensure the system is behaving appropriately by adding constraints, such as the expected behaviour of traffic lights (changing through red, amber, green), for example.
Iain Whiteside noted that autonomous vehicles might well fail to recognise unusual dangerous situations. He gave the illustration of a cyclist, wearing headphones in the dark and with a large mirror strapped to her back, and commented that “safety is heterogeneous”. Taking a deep perspective, he noted that the code for neural networks is in a sense simple, involving matrix multiplication: the “bugs” are really problems with the data. He noted the tension between accuracy vs the time and data needed to train systems. Iain also recommended testing autonomous vehicles using scenarios, or safety cases, drawing attention to the valuable resource for researchers in AI Validity given at https://nsc.nasa.gov/resources/case-studies produced by NASA, which contains case studies of how systems fail.
Michael Bronstein pointed out that the success of deep learning was a consequence of the abandonment of the universal approximation, and its replacement by the imposition of translational invariance. He stressed the adaptation of this idea to graphs through local permutation invariance. He used the Netflix recommender system to illustrate graph theoretical ideas, looking at various challenges, such as how to update as new data comes in.
Pushmeet Kohli opened by asking why it is that image classifiers can perform excellently on one image data set, but fail miserably on a different one: with this question in mind, he discussed the need for rigorous training, robust AI, and the verification of AI systems, stressing the need for insensitivity to meaningless changes in the data, and noting how 8 slight changes to the system can dramatically reduce performance. He described strategies for tackling such problems, for example using adversarial systems. He commented that an approach to randomly generate examples to improve robustness does not always work because this approach may not capture all sensitive scenarios. To cope with this, he and his colleagues are working on mathematical approaches based on buffer zones of neighbourhoods around points to be classified. Pushmeet stressed the need for rigorous testing, adding that AI software needs to conform to social norms for safety and values, through the formal specification of robustness, fairness and compliance. He discussed “explainability” and the need to consider who requires the explanation and what needs to be explained. He distinguished between explaining a learnt system and learning an explained system, and noted that the natural inductive biases in human language means that explainable models can generalise better than unexplainable models – provided those biases regularise in the right way. He also noted the fundamental point that the challenges were not merely technical, but were also social.
Giles Herdale described the so-called Peelian Principles, named after Sir Robert Peel, and in particular the idea that “the police are the public and the public are the police”. He discussed drivers for data-driven policing, and the challenge posed by the explosion in digital evidence and changing offending patterns. In the context of these challenges, he discussed independent oversight, internal management and expertise, and public engagement. He also drew attention to the critical questions of what the public are seeking to achieve and of how to measure success.
Stephanie Hare commented on the evolution from first generation biometrics (e.g. fingerprint, DNA) to second generation (e.g. face, eye, gait recognition), stressing the critical importance of joining up data, and noting that the police need a legal framework for the use of technology. In particular, she highlighted issues of privacy and misuse of data in AI systems. In many parts of the world, including UK and other western countries, second generation biometrics can be taken without consent or even knowledge: there is currently no legal protection. Many police forces and legislatures across the world are now reviewing this situation
The panel (consisting of Carly Kind, Tom Smith, Zeynep Engin, Marta Kwiatkowska and Martin Goodson) discussed the explosion of interest in AI ethics and the centrality of ethics in AI development. The nature of ethical failures in the AI world was contrasted with that in (for example) engineering. It was suggested that further discussion of ethics in AI was not required – we know what we need to do – what is required is development of appropriate methodology to ensure the ethical requirements are implemented and validated correctly. An analogy was drawn with clinical trials in medicine which is strongly regulated: perhaps a similar approach is required for AI? The question was raised about whether a machine learning system can be considered flawed because it encodes discrimination and sociodemographic inequalities found in society. There was a general consensus that, no, it couldn’t, but we have the opportunity to build machine learning systems that support rectifying the problems of discrimination in society. Care is needed when dealing with this question since discrimination in machine learning may in fact be a consequence of bias in the training data. Other topics discussed by the panel included: the extent to which we should seek to encode emotions in AI systems, the fact that much knowledge is tacit and data-based (e.g. how anaesthetics work), the use of AI systems in accountancy, the perennial issue of models encapsulating data bias, whether we should 9 hold AI systems to a higher level of accountability than humans, the fact that “data science” is not (yet) a profession, the suggestion that most ethical breaches occur because people are not properly trained, the challenge of breaking down barriers between technical and policy communication, and the question of how things will change in the future, with the practical comment that maybe we have to accept the sub-optimal for now, knowing it will improve in time.
The Ada Lovelace Institute is an independent research and deliberative body, supported by the Nuffield Foundation, with a mission to ensure that data and AI work for people and society. We believe data and AI must be designed, deployed and governed in ways that reflect social justice values to generate positive outcomes for societies and people. We work to better inform policy, regulation, standards and technical approaches so that data and AI can be put to the service of humanity, and not the other way around. We are delighted to support the Validate AI conference and promote informed debate about data-driven technologies. We believe that debate which convenes diverse voices is essential to ensuring these technologies
generate positive outcomes for societies and people, and that those benefits are justly and equitably distributed to enable social and individual well-being.
The OR Society is the home to the science + art of problem solving. Through practice, research and teaching, our community helps individuals and teams thrive in their analytical careers. We are a vibrant, supportive community of professional problem solvers, a place for analytical thinkers to learn, share, optimise and excel in their career. Our role is to champions the value of Operational Research in the modern world, through the coordination, orchestration and curation of knowledge and understanding. We were delighted to sponsor the Validate AI conference as the advancement of how we validate and implement robust AI systems into organisational processes is consistent with the society’s objectives to
promote analytical rigour.
Imperial College London is the only UK university to focus entirely on science, engineering, medicine and business. Our international reputation for excellence in teaching and research sees us consistently rated in the top 10 universities worldwide. Our research makes a demonstrable economic and social impact through the translation of our work into practice worldwide. We aim to provide an education for students from around the world that equips them with the knowledge and skills they require to pursue their ambitions. The Data Science Institute at Imperial College is proud to support the Validate AI conference, which is aligned with our mission to foster, advance and promote excellence in data science research and education across all domains for the benefit of society.