The Alignment Problem: Machine Learning and Human Values By Brian Christian


Trying to understand alignment of human values and machine learning
It’s a great eye opener and easy to understand covers a lot of ground The Alignment Problem: Machine Learning and Human Values

A jaw dropping exploration of everything that goes wrong when we build AI systems and the movement to fix them.Today’s “machine learning” systems, trained by data, are so effective that we’ve invited them to see and hear for us―and to make decisions on our behalf. But alarm bells are ringing. Recent years have seen an eruption of concern as the field of machine learning advances. When the systems we attempt to teach will not, in the end, do what we want or what we expect, ethical and potentially existential risks emerge. Researchers call this the alignment problem.Systems cull résumés until, years later, we discover that they have inherent gender biases. Algorithms decide bail and parole―and appear to assess Black and White defendants differently. We can no longer assume that our mortgage application, or even our medical tests, will be seen by human eyes. And as autonomous vehicles share our streets, we are increasingly putting our lives in their hands.The mathematical and computational models driving these changes range in complexity from something that can fit on a spreadsheet to a complex system that might credibly be called “artificial intelligence.” They are steadily replacing both human judgment and explicitly programmed software.In best selling author Brian Christian’s riveting account, we meet the alignment problem’s “first responders,” and learn their ambitious plan to solve it before our hands are completely off the wheel. In a masterful blend of history and on the ground reporting, Christian traces the explosive growth in the field of machine learning and surveys its current, sprawling frontier. Readers encounter a discipline finding its legs amid exhilarating and sometimes terrifying progress. Whether they―and we―succeed or fail in solving the alignment problem will be a defining human story.The Alignment Problem offers an unflinching reckoning with humanity’s biases and blind spots, our own unstated assumptions and often contradictory goals. A dazzlingly interdisciplinary work, it takes a hard look not only at our technology but at our culture―and finds a story by turns harrowing and hopeful. The Alignment Problem: Machine Learning and Human Values

The activity inside a computer is nothing than a changing series of high and low voltages (1's and 0's). Computers don't understand anything: a garbage can rolling thru a road intersection is the same as a mother pushing a baby carriage. A computer must be laboriously taught to make the distinction , and the programmer may not omit anything important. The book spends allot of time describing how programmers have tried to accomplish that.

It seems humans can't help but project their emotions and desires onto an inanimate machine. The brain is not a computer. Who is the user? Try to answer that question and you find yourself in an infinite regression. There is no mind substance apart from the brain that can be algorithmically described. There is indeed a question of human rights and dignity, but the first problem is that so many people can't get beyond their fervent wishes and have a very limited understanding of what they are doing. The Alignment Problem: Machine Learning and Human Values Like the works of Steven Pinker and Daniel Dennett, this book explains cutting edge ideas in psychology, AI, and philosophy ideas that are having a big impact on the world.

AI and Machine Learning made huge advances in the last decade and billions of dollars flowed into R&D. This book is about the collision of new AI technologies with the human world. People want AI systems to make decisions that are reliable, understandable, and responsive to human morality and fairness. Is such “human aligned” AI possible, and what ideas do we need to achieve it?

The book narrates and explains many seminal developments in AI, including:
The birth of neural networks in the 1950s and their dramatic renaissance and take off since 2009 as “Deep Learning”
Rich AI models for the meaning of words (“word vectors”)
Connections between motivation systems in the brain (behaviorism, dopamine) and Reinforcement Learning in AI
How Google DeepMind used neural nets to learn to play Atari and to defeat the world champion at Go
How OpenAI, founded by Elon Musk to democratize AI, pioneered techniques for training neural networks directly from human feedback.

I work in this field, and I found these explanations engaging, accessible, and scientifically accurate. The author also does a great job of telling some of the human stories behind these developments.

Full disclosure: As well as being a researcher on AI alignment, I am a friend of the author. I was interviewed as part of the research process for the book. The Alignment Problem: Machine Learning and Human Values I am a voracious reader, or should I say, listener because nowadays, I first listen to books which may be of interest. More often than not I find that popular business and even science books are large in volume but thin in content.
The big exception is Brian Christian’s book “The Alignment Problem”, 1st Edition (October 6, 2020). Briefly, Christian defines the alignment problem as arising from our imprecise or incomplete instructions to artificial intelligence which may result in a potentially catastrophic divergence from what we really want and what AI actually does. Alignment ensures that models capture our norms and values so that they implement what we mean, intend and want.
Christian’s narrative, always compelling and interesting, opens up new insights on every page even if you are generally familiar with the field. The book extends over 334 pages and is then followed by an 117 page notes and bibliography section. In other words, this huge accomplishment of a book is of value to the academic researcher in AI, ethicists and legal scholars, the advanced analytics practitioner and their managers, and an interested lay audience. It should be noted that the book is also of particular interest to physicians who are currently developing or using AI algorithms for diagnosis or patient management.
Artificial intelligence is currently on the top of its own hype cycle again. In contrast to previous decades we now see an avalanche of real world applications covering up the potential for failure: obvious, patent, and potentially fixable, on one hand, and latent, hidden or lurking, waiting to trigger catastrophe, on the other hand. Raising awareness now will help avoid serious mistakes. The Alignment Problem: Machine Learning and Human Values I've been a huge fan of Brian Christian's writing since well, since before 'The Most Human Human,' but that's where he truly proved himself. Reading Brian's work is like going to a cocktail party that you weren't totally sure you wanted to go to, then finding yourself having spent the last three hours spellbound by the bald man in the corner, reducing incredibly complex ideas to meaningful and entertaining analogies that, by the end of the night, have completely transformed your worldview. Truth be told, I'm a little nervous to dive deeper into the issues lurking in the future of AI at this point. But if there's anyone I trust to enlighten me on high stakes issues without crushing my soul (or my brain), it's Brian Christian. The Alignment Problem: Machine Learning and Human Values I read a lot of books written about AI. It is a topic my friend and I discuss for hours. She actually met Stuart Russell right before the pandemic and he signed his book for her. The Alignment Problem is another great book in the study of AI. The book is easily understood by the laymen like she and I. I did read it slow so I can think about the topics as he presents them. How to program AI to do what we want it to do is going to be a major consideration over the next decade and century. We must align what the machine is asked to do in relation to what we as a society and the programmers desire the machine to do.

Many examples are given about the history of Machine Learning, the early attempts at programming machines to play games for example. There are as well, many examples that went wrong and how the programmers decided to solve the problems. The author also, talks about various ways people and machines learn and how this is being considered in the alignment problem.

This is a book I will reread many times and follow over the next few years how AI develops. Great book one i recommend for everyone. The Alignment Problem: Machine Learning and Human Values

The

Free read The Alignment Problem: Machine Learning and Human Values