August 25, 2021

Assistant Professor at NEU Khoury

I am thrilled to announce that I will be joining the Northeastern University Khoury College of Computer Science as an Assistant Professor in Fall 2022.

For prospective students who are thinking of a PhD, now is a perfect time to be thinking about the application process for 2022. Drop me a note if you have a specific interest in what our lab does. And if you know somebody who would be a fit, please share this!

http://davidbau.com/research/
https://www.khoury.northeastern.edu/apply/

We think that understanding the rich internal structure of deep networks is a grand and fundamental research question with many practical implications. (For a talk about this, check out my PhD defense). If this area fascinates you, consider applying! The NEU Khoury school is in downtown Boston, an exciting, international city, and the best place in the world to be a student.


Posted by David at 06:48 PM | Comments (1)

August 24, 2021

PhD Defense

Today I did my PhD defense, and my talk will be posted here on youtube. Here is the talk!

Title: Dissection of Deep Networks

Do deep networks contain concepts?

One of the great challenges of neural networks is to understand how they work. Because a deep network is trained by an optimizer, we cannot ask a programmer to explain the reasons for the specific computations that it happens to do. So we have traditionally focused on testing a network's external behavior, ignorant of insights or flaws that may hide within the black box.

But what if we could ask the network itself what it is thinking? Inspired by classical neuroscience research on biological brains, I introduce methods to directly probe the internal structure of a deep convolutional neural network by testing the activity of individual neurons and their interactions.

Beginning with the simple proposal that an individual neuron might represent one internal concept, we investigate the possibility of reading human-understandable concepts within a deep network in a concrete, quantitative way: Which neurons? Which concepts? What role do concept neurons play? And can we see rules governing relationships between concepts?

Following this inquiry within state-of-the-art models in computer vision leads us to insights about the computational structure of those deep networks that enable several new applications, including "GAN Paint" semantic manipulation of objects in an image; understanding of the sparse logic of a classifier; and quick, selective editing of generalizable rules within a fully trained StyleGAN network.

In the talk, we challenge the notion that the internal calculations of a neural network must be hopelessly opaque. Instead, we strive to tear back the curtain and chart a path through the detailed structure of a deep network by which we can begin to understand its logic.


Posted by David at 03:13 PM | Comments (0)

August 20, 2021

Antivax, Antiglyphosate

Where does the antivax sentiment come from? Does it seem bizarre to you?

If the anti-vaxxers seem crazy, here is a question for you: how do you feel about the Monsanto herbicide glypohsate? Do you have the impression that it might be carcinogenic?

Now take a read of a literature review that tries to dissect why IARC judges glyphosate as carcinogenic while the EU does not. Take a look at the underlying studies. Think about which studies are causal, and which are retrospective only. Think about whether results have biological plausibility. A lot of different studies have been done. Not every study is well-reasoned, and some studies draw conclusions that are not well-supported by their own evidence. Do you feel you can read the research critically? And even after making a scientific judgement, would you still worry about spraying glyphosate all over your own weedy yard? Would you let your kids run around in it?

I happen to not worry about glyphosate - it is familiar to me, and the evidence seems to indicate it is much less dangerous than other common agents I use, like bleach. But many of my very scientific friends do worry about it. They wouldn't let their kids touch it, no matter how many studies have been done injecting the stuff and feeding it to animals.

I think it's important to acknowledge that we can be fearful even if is not rational. We all have it: there is a little bit of that anti-unknown, anti-chemical, anti-novelty fearful sentiment in us all.


Posted by David at 06:05 AM | Comments (1)

August 06, 2021

Global Catastrophizing

Do you think the world is much darker than it used to be? If so, you are not alone.

I have always assumed that a feeling of psychological decline is just a side-effect of getting older. But a paper by Bollen, et al., out in PNAS today suggests that a darker outlook in recent years might not be specific to any of us individually. By analyzing trends in published text in the Google ngrams corpus, researchers from Indiana University and Wageningen have discovered that across English, Spanish and German, text published in the world shows sudden changes in language use over time that are indicative of cognitive distortions and depression, coinciding with major wars or times of social upheaval.

The chart above is from Bollen's paper, and it counts something very simple. For every year, it counts how many times a particular set of short phrases appear in the printed books published in that year. The annual counts come from Google's Books ngram corpus - derived from scans of published books - and the 241 phrases counted are word sequences chosen by a panel of cognitive behavioral therapy specialists as markers for cognitive distortion schemata (CDS). That is, they are phrases that would suggest systematic errors in thinking that are correlated with mental health issues that are treated by psychologists.

For example, one of the 241 counted phrases is "you always," because those words often indicate overgeneralization such as in the sentences "You always say no," or "You always win." The bigram "everyone knows" indicates mind-reading, because it reveals that the speaker has a belief that they know what other people are thinking. The trigram "will never end" indicates catastrophizing, an exaggerated view of negative events. In total the panel of experts cataloged each of the 241 selected phrases, as a marker for a dozen specific cognitive distortions. These cognitive distortions are correlated with depression, so it is interesting to ask whether large-scale trends in the usage of these phrases reveals mass changes in psychology over time.

The chart suggests that it might. It seems to reveal suffering in Germany coinciding with World Wars I and II, and trauma in the English-speaking world at the Spanish-American War, Vietnam war protests, and 9/11. Strikingly and worryingly, all the languages show a dramatic increase in cognitively distorted use of language since 2007. If you believe the linguistic tea leaves, our collective state of mind seems to have taken an extraordinary turn for the worse in the last decade — globally.

The paper is an example of a use of the Google Books ngrams corpus. This is a pretty great resource that catalogs language use by counting words and ngrams in about 4% of all published text, by year, and it means that you can easily look further into the data yourself. The authors provide their list of phrases, so you can examine the trends by individual category and phrase. Here are the top phrases for catastrophizing in English:



Explore and modify the query yourself here. You can see spikes in certain phrases corresponding to WWI and WWII, and the upwelling, in recent decades, of expressions of the idea that a variety of things "will end" and "will not happen".

It is when the 241 phrases are added together, when we see dramatic recent spikes that are reminiscent of the climate change hockey stick plot by Mann, Bradley and Hughes.

Do you agree with the authors that these changes in word usage are meaningful? Have we been experiencing a catastrophic worldwide decline in psychological health since 2007?

Or is this just an example in which the authors themselves are catastrophizing, looking at data in a way that interprets events in the world as much worse than they actually are?

Previous musings on society-wide catastrophizing here.


Posted by David at 08:10 AM | Comments (0)

March 18, 2021

Passwords should be illegal

As part of modernizing U.S. infrastructure, America should eliminate passwords.

Our use of passwords to build security on the internet is akin to using flammable materials to build houses in densely-populated cities.  Every single website that collects, stores and transmits password invites a new cybersecurity catastrophe.

When half of Chicago burned down in 1871, citizens reflexively blamed the disaster on evil actors: arsonists, immigrants, communists.  After the fire, the first response of political leaders was to impose martial law on the city to stop such evil-doers.  From our modern perch, it seems obvious that the blame and the fix was misplaced.  Even if the spark were lit by somebody with bad intentions, the scale of the disaster was caused by outdated infrastructure.  Chicago had been built out of combustible materials that were not safe in a densely-built city.

Our continued use of passwords on the internet today poses the same risk.

Just as a small fire in a flammable city can turn into a massive disaster, on the internet, a single compromised password can lead to a chain reaction of compromised secrets that can open vast parts of the internet to hacking.  The fundamental problem is that we store and transmit many of the secrets that we use to secure the internet, including passwords, on the internet itself.

In the 2020's using, transmitting, and storing passwords on the internet should be as illegal as constructing a Chicago shanty out of incendiary cardboard.

Physical key-based authentication systems are cheap.  They keep secrets secure on computer chips that are not connected to the internet and that never reveal their secrets on the network.  If physical keys were used everywhere we currently use passwords, all internet hacking would be far harder and slower.

Key-based login systems have been available for decades, but because standards are not mandated, they are adopted almost nowhere.  Physical keys are slightly more inconvenient for system-builders, and consumers do not demand them because the dangers of hacking are invisible.  It is an excellent example of a situation where change is needed, but the marketplace will not create the change on its own.

That is why our country's best response to the increasing wave of hacking disasters should be led by people like the folks at NIST, rather than the U.S. Army.  We should standardize, incentivize, mandate, and fund the use of non-password based authentication in all computer systems over the next few years. A common set of standards should be set, so that people can log into all systems using trustworthy physical keys that cannot be hacked remotely.

Eliminating passwords would make more of a difference to cybersecurity than any clever retaliation scheme that the cybersecurity soldiers might cook up.  Although there are certainly evil actors on the internet, we ourselves are the ones who empower hackers by perpetuating our own dangerous practices.

As we modernize U.S. infrastructure, we should prioritize modernizing standards and requirements around safe authentication systems on the internet.



Posted by David at 12:28 PM | Comments (2)

October 16, 2020

Deception is a Bug

Today Twitter and Facebook decided to manually limit the spread of the NY Post's unverified story about a hack on the Biden family. Taking responsibility for some of the broad impacts of their systems is an excellent move.

But the fact that FB+Twitter needed to intervene is a symptom of badly flawed systems. We all know that the systems would have otherwise amplified the misinformation and caused widespread confusion. In other words, we all know our big social networks have a bug. It is a fundamental bug with ethical implications - but in the end, it is a bug, and as engineers we need to learn to fix this kind of issue. As a field, we need to be willing to figure out how to design our systems to be ethical. To be good.

What does it mean for an AI to be good?

The fundamental reason Twitter and Facebook and Google are having such problems is that the objectives used to train these systems are wrong. We can easily count clicks, minutes of engagement, re-shares, transactions. So we maximize those. But we know that these are not actually the right goals.

The right goal? In the end, a system serves users, and so its purpose is to expand human agency. A good AI must help human users make better decisions.

Yet improving decisions is quite a bit harder than maximizing page views. It requires getting into subtle issues, developing an understanding of what it means to be helpful, informative, honest. And it means being willing to take on tricky choices that have traditionally been the realm of editors and policymakers. But it is possible. And, as a field, it is what we should be aiming for.

A few more thoughts in previous posts:
The Purpose of AI
Volo Ergo Sum


Posted by David at 12:42 PM | Comments (1)

August 19, 2020

Rewriting a Deep Generative Model

Can the rules in a deep network be directly rewritten?

State-of-the-art deep nets are trained as black boxes, using huge piles of data and millions of rounds of optimization that can take weeks to complete. But what if we want change a learned program after all that training is done?

Since the start of my PhD I have been searching for a way to reprogram huge deep nets in a different way: I want to reconfigure learned programs by plugging pieces together instead of retraining them on big data sets. In my quest, I have found a lot of cool deep net structure and developed methods to exploit that structure. Yet most ideas do not work. Generalizable editing is elusive. Research is hard.

But now, I am delighted to share this finding:

Rewriting a Deep Generative Model (ECCV 2020 oral)
Website: https://rewriting.csail.mit.edu
Code: https://github.com/davidbau/rewriting
Preprint: https://arxiv.org/pdf/2007.15646.pdf
2 min video: https://www.youtube.com/watch?v=i2_-zNqtEPk


Editing a StyleGANv2 model of horses. After changing a rule, horses wear hats!

Unlike my GanPaint work, the focus of this paper is not on changing a single image, but on editing the rules of the network itself. To stretch a programming analogy: in previous work we figured out how to edit fields of a single neural database record. Now we aim to edit the logic of the neural program itself.

Here is how: to locate and change a single rule in a generative model, we treat a layer as an optimal linear associative memory that stores key-value pairs that associate meaningful conditions with visible consequences. We change a rule by rewriting a single entry of this memory. And we enable users to make these changes by providing an interactive tool.

I edit StyleGanV2 and Progressive GAN and show how to do things like redefine horses to let them wear hats, or redefine pointy towers to have buildings that sprout trees. My favorite effects are redefining kids eyebrows to be very bushy, and redefining the rule that governs reflected light in a scene, to invert the presence of reflections from windows.

Here is a 10 min video:
https://rewriting.csail.mit.edu/video/

MIT news has a story about it here today:
http://news.mit.edu/2020/rewriting-rules-machine-generated-art-0818


Posted by David at 11:11 AM | Comments (1)

July 05, 2020

David's Tips on How to Read Pytorch

Pytorch has a great design: easy and powerful. Easy enough that it is definitely possible to use pytorch without understanding what it is doing or why. But it also gets better the more you understand.

As part of summer school at MIT, next week I'm doing a lecture to introduce students to pytorch. I have written a few code examples that I hope will give students a head start on understanding the design of pytorch. Each concept is illustrated visually with a cute minimal hackable example. All the examples are notebooks that are hosted on Google Colab.

It covers tensor indexing conventions, benchmarks gpu versus cpu speeds, explains autograd with simple systems, and plots what optimizers are doing using 2d problems. Then I put the pieces together with a detailed discussion of network modules and data loaders, training toy networks where the whole space can be visualized as well as a simple but realistic five-minute ResNet training example.

Here are David's Tips on How to Read Pytorch.


Posted by David at 02:58 PM | Comments (3)

April 25, 2020

A COVID Battle Map

Whenever Heidi gets a headache after coming back from the hospital, I worry about losing her to COVID.

But I am very aware that, with the virus already so widespread, the decisive battle is no longer being fought by doctors in the hospitals. They are just buying time, containing the threat just like you and I do when we social distance.

The outcome will depend on a race between two global teams furiously trying to hack a dozen proteins. The good guys are thousands of biologists, an historic worldwide collaboration. The bad guys are the random forces of natural selection, the mutations that happen inside each carrier. Thanks to the Bedford lab at Fred Hutchinson, you can see a map of the battlefield here, tracing the random moves made by the bad guys: (data from GISAID)

What are the bad-guy mutations doing? A small study came out of Zhejiang university this week (medrxiv, not peer-reviewed) that hints at the risks as we let the virus propagate and evolve. They did cell-culture studies on 11 samples and found, for example, a 19-fold difference in cell-culture virulence between one version similar to the virus in WA, CA, OR, and VA (not very virulent) compared to one resembling strains in England and France (far more virulent). One of the versions from Wuhan was 249 times worse. (Strains common in NY or Italy were not included.)

So as we celebrate that WA state seems to be beating the virus, this study highlights that WA has just beaten one strain. The European strains spreading elsewhere are different and might actually be more deadly. I think is important to contain covid before an even-worse strain spreads, as happened in 1918.

Happily, in 2020, we can map out a set of weak points that the good guys can counterattack. Here is a survey paper. Some notable targets:

  1. Attacks on the ACE2 receptor, the molecular passkey used by the virus to break into human cells.
  2. Attacks on the viral replication machine, the intricate RNA-dependent-RNA polymerase RdRps/NSP12.
  3. Attacks on a key link in the viral factory, the protease 3CLpro/NSP5 that cleaves out the viral proteins after they are made in one big chain.
  4. Old-fashioned attacks on the virus armor. Vaccines target the S protein shell on the outside of the virus.
  5. New-fangled behind-enemy-lines attacks by CRISPR hotshots who want to directly chop up viral RNA.
  6. Some scientists are working on defenses that improve the human body's response, steeling our organs to viral attack by trying to calm the inflammation that causes such problems.
  7. Others are working on defenses that transplant a more robust immune response, via donated plasma.

The New York Times has beautiful renderings of all the molecular attack targets.

Unlike in a shooting war, we do not have news reporters going into the battlefield to report on the days wins and losses. But maybe we should. None of these is sure thing. But they all have a chance, and there are real salvos being launched on each of these targets.

On both sides, the battlefield is active.

on facebook here


Posted by David at 06:45 PM | Comments (1)

March 25, 2020

COVID-19 Chart API

Here is the COVID-19 Live Chart API. Use it to create a custom live chart of COVID-19 stats on a linear or logarithmic scale, comparing the set of countries and states that you choose (or an automatically sorted set of worst states or countries), on the timeframe that you want to see.

New 3/27/20: You can now plot local data of most US Counties. Just type the counties, states, and countries you want to see into the search box, and you can make a custom graph focused on the localities you care about.

It is designed to help you use current data to anticipate the future. Click on "advanced options" on covid19chart.org. It just takes a few clicks to make a new visualization.

Once you have created a custom chart, you can email it or print it for your local policymaker. Or better yet, if you are making a dashboard that leaders will see every day, theme the graph dark or light to match your webpage, use the "bare" mode for embedding it as an iframe, like this:

<iframe src="https://covid19chart.org/#/?start=%3E%3D50&include=WA%3BMA%3BNY%3BItaly&top=0&domain=Intl&theme=dark&bare=1" width="500" height="388"></iframe>


(The embedded chart is interactive.)

The data is live, pulled directly off Johns Hopkins CSSE COVID data feed on github. Although that feed is in flux and changes format every few days, I will track their changes and the chart up-to-date as needed. Please email me (david.bau@gmail.com) if you have any problems with this API.

The current data tell a simple story.

In the US, if we want to avoid a grim future, we need to be making better decisions now. Every state of our country is seeing a similar exponential explosion, just starting on different days. Please use these charts to tell this story. And thank you for helping our leaders understand the importance of our choices today.

Continue reading "COVID-19 Chart API"
Posted by David at 10:46 AM | Comments (5)

March 24, 2020

The Beginning

Today marks the beginning of the COVID-19 crisis for me. It is the first day that surgeons are being called in from their regular duties to take care of the wave of COVID-19 patients at MGH. Heidi needs to run into the hospital. We will have weeks or months of this ahead.

I am terrified.

The COVID-19 chart has been updated to include both state-level and international statistics, and it includes an API so that you can make, link, and embed a custom chart that focuses on the states or countries of your choice. The (no doubt stressed-out) CSSE team has been screwing up the data feed, but I will keep the data cleaned and correct on the live chart as long as it can be patched together. Below we can see America first in the chart today.

Please use it as a tool to pressure your local policymakers to take this crisis seriously.

Thank you.


Posted by David at 10:40 AM | Comments (1)

March 23, 2020

No Testing is not Cause for Optimism

Two readings and a thought related to covid-19 testing.

Lack of information requires us to believe two contradictory things at once. From a policy point of view, we need to understand that very few people are infected yet. And from a personal behavior point of view, we need to understand that many people are already infected.

Policy first. Some people think that the lack of testing means that there could be far more asymptomatic cases than we know, and therefore the disease could far less deadly than we imagine.

But consider the case of the town of Vò, near the epicenter of the Italian outbreak, where all 3000 residents were tested. As severe as the outbreak is in Italy, it corresponded to less then 3% of the population being infected. So as bad as the Italian case is, at least in the one town that was tested, it could be 30 times worse. Blindness is not cause for optimism.

Which individuals should be tested? The right behavior is to do the things that maximize lives saved. That means testing should be done in situations where it would change care, for example on on healthcare providers who do not have the option to isolate, so that they do not inadvertently spread it to other providers and patients.

But of course that means many infected people will be untested, so everybody needs to operate under the assumption that we are all infected.

Paradoxically this lack of information means we need to keep in mind two different realities at once. First, we need to recognize that almost nobody has it yet, so the society-wide damage can and will get far far worse; and second, that we and others are likely to have it, so our personal risk and responsibility is very high. We need to isolate.

The parable of two realities corresponds to the logarithmic and linear view of the disaster. I have posted an updated version of the covid-19 time series tracker, which provides both views on covid19chart.org.


Posted by David at 01:45 PM | Comments (0)

March 22, 2020

Two Views of the COVID-19 Crisis

I have posted an interactive chart of USA COVID-19 cases.

This chart lets you see coronavirus data from two different points of view: the policymaker's view, and the doctor's view.

For policymakers, the chart lets you see USA data in the same way the Financial Times COVID-19 plot by John Murdoch compares policies internationally. Select the logarithmic totals with a '>=100' starting threshold, so that "day zero" is the first day there are 100 cases in a state. Over time, if different states' policies have different effects on the growth of the virus, the exponents, and therefore the slopes, will reveal the differences.

The other point-of-view is the doctors-eye view. Doctors must deal with the patients who walk into the ER and who who lie sick in ICU beds. To anticipate these numbers, switch to the 'delta linear' view in the current month. The spikes show why the panic is justified, and why minor policy changes have massive ramifications.

The takeaway? The chart re-emphasizes the point that this is not a game. There is a huge gap between the "policymaker's" view and the "doctor's" reality on the ground. Slight changes from a policymaker's point of view have massive ramifications for doctors.

After our leaders negotiate about a "gradual" shutdown of car factories, Michigan illnesses explode. After beaches stay open for one last lucrative spring break party, Florida cases skyrocket. And what begins as a local outrage will become a healthcare shortage, then a nationwide menace. A single idiotic master of the universe could trigger an outbreak that will use up the ventilators that would have saved your grandfather.

In our 50 states we are all linked. Despite dramatically different local policies, it is likely that our rate of infection growth will be largely the same across the country. In coming days, this chart will tell the story of our national interconnectedness.

Please. We need to take the crisis more seriously than we are. Our corporate, city, state, and federal leaders are not doing enough. "Social distancing" of the coastal elite needs to give way to a much more universal regime of physical isolation, enforced shutdowns, shifting of priorities, deferral of debts, and testing, testing, testing, nationwide.

The graph automatically updates every day based on current data. Please share. And please isolate.

I made the chart to help Heidi (who is a surgeon at MGH) see summaries of some USA statistics that are not being plotted in the media. The code is open-source at github.com/davidbau/covid-19-chart. It is just a bit of HTML and JS, and should be easy to extend to show more information. Pull-requests are welcome.


Posted by David at 01:57 PM | Comments (3)

January 12, 2018

The Purpose of AI

What does it mean for an AI to be good?

Is Omniscience Good?

There are benefits to having computer systems that know everything. For example, yesterday a friend recounted a story about leaving a laptop in a taxi in China. Local police stations in China have a system that can call up any recorded video anywhere in the city, so they used the approximate time of the taxi ride obtain to a video clip of the exact moment of the cab pickup. Soon, they had the plate numbers and called the driver, who promptly found the laptop and returned it to its owner. Today, routine total surveillance in China is coupled with AI systems that constantly sift through the vast stream of data to identify and track every individual person, catalog every interaction, and flag anomalous behavior.

This makes prosecuting crime very easy in China. The court will be presented with a video tape summary of footage of the accused in the hours and days before and after the crime. AI systems, connected to a total surveillance apparatus, are able to automate weeks of police work and create a narrative about why a person is guilty. The same systems also simplify the hard work of putting a rapid stop to uncomfortable social disruptions such as demonstrations and protests.

China has no fourth and first amendments to give them pause, and so that country gives us a glimpse of what is possible with widely available technology today. And maybe it is a picture of humanity's future everywhere. Quiet streets, low crime, no picketing. Never lose a laptop again.

Is that a good thing?

The Purpose of AI

In our pursuit of making AI systems that are more accurate, faster, and leaner, we risk losing the sight of the fundamental design question: What is it for? The systems that we build are complex, with multiple intelligent agents interacting in the system, some human, and some not. So to understand the design of the whole system, we must ask, what is the role of the human, and what is the role of the AI?

Both humans and AI can perceive, predict, and generalize, so there is sometimes a misperception that the two roles are interchangeable. But they are not. Humans stand apart because their purpose is to be the active agents, the deciders. If that is the case, then what is the role of the AI? Can an AI have agency?

There are two forms of interaction between AI behavior and human behavior where agency seems messy.

  • AI can predict human behavior.
  • AI can shape human behavior.

The problem with optimizing a system around these two design goals is that they presume no role for human agency. It is assumed that a good system will make more accurate predictions - for example the way that Facebook is very good at predicting which thing you will click on next. And it is assumed a good system will be more effective at shaping future behavior - for example the way Google is very good at arranging advertising in a way that maximizes your likelihood of clicking on it.

If a system is designed around those principles alone, the humans are just treated as a random variable to be manipulated, and there is no decision maker in the design. These designs are incomplete. Like any engineered system, an AI is always designed for some purpose. When we do not consider that purpose, the actual decision makers have been erased from the picture.

The proper purpose of an AI is this:

  • AI should amplify human decisions.

What a Good AI Does

The question of AI goodness comes down to how we can evaluate whether an AI is good or not. We cannot stop at evaluating merely whether an AI is more accurately predictive, or whether it is more effective in achieving an outcome.

We need to be transparent about answering the questions:

  • Who is the user?
  • What decisions do they make?
  • Are the decisions improved by the AI?

For example, with the Chinese surveillance system, the people being observed by the cameras are not making any decisions that are improved by the AI. The people on the street are not the users. The actual users are the people behind the console at the police station: they are the ones whose agency is amplified by the system. They use the system to help decide what to look at, who to call, and who to arrest. To understand whether the AI is good, we need to understand whether it is serving the right set of users, and whether their decisions are improved. That means beginning with an honest discussion of what it means for a police officer to make a good decision. The right answer is likely to be more complicated than a question of crime and punishment.

Most of us work on more prosaic systems. Today I spoke with a researcher who is applying AI to an educational system. She has a large dataset of creations (student programs) made by thousands of students, and she wants to make suggestions to new students about what pieces (program statements) they might want to include in their own creations. In her case, the target user is clearly the student making the creation, and the system is being optimized to predict the user's behavior.

However, the right metric is not predictive accuracy, but whether the user's decisions are improved. That gets to a more difficult discussion of what it means to make a good decision. For example, if most students in her data set share a common misconception about the subject being learned, then the system will optimize predictive accuracy by propagating the same misconception to future students. But that does not amplify the agency of users; it does not improve decision making. Instead, it is exactly the type of optimization that results in an AI that will dull the senses of users.

This is the same problem being faced by Facebook and Google today. Misconceptions, lazy decision making, and addictive behavior are all common human phenomena that are easy to predict and trigger, and so when we optimize systems to maximize accuracy and efficacy in their interactions with humans, the systems solve the problem by propagating these undesirable behaviors. The AI succeeds in solving its optimization by robbing humans of their agency. But this is not inevitable: AI does not need to dehumanize its users.

Building Good AI is Hard

To build good AI, it is not enough to ask our AI to merely predict behavior or shape it reliably. We need to understand how to create AI that helps amplify the ability of human users to make good decisions. We need to understand who the users are and what decisions they make.

In the end, building a good AI means building an authentic understanding of what it means to make a good decision.


Posted by David at 09:14 AM | Comments (0)

December 19, 2017

npycat for npy and npz files

Pytorch, Theano, Tensorflow, and Pycaffe are all python-based, which means that I end up with a lot of numpy-based data and a lot of npy and npz files sitting around my filesystem. All storing my data in a way that is hard to print out. (Why this format?)

Do you have this problem? It is nice to pipe things into grep and sed and awk and less, and, as simple as it is, the npy format is a bit inconvenient for that.

So here is npycat, a cat-like swiss army knife for .npy and .npz files.

>  npycat params_001.npz
 0.46768  2.4e-05 2.03e-05  2.3e-05   ...   2.4e-05  7.4e-06  5.1e-06  4.5e-06
 2.4e-05  0.46922   0.0002  1.2e-05   ...   5.2e-05  5.9e-05  2.7e-05  5.3e-06
 2.6e-05  0.00026  0.59949  8.3e-05   ...   7.4e-06  5.6e-05  5.9e-06  1.3e-05
  ...
 1.1e-05 8.59e-05  6.4e-05 9.74e-05   ...     2e-05  0.68193  2.2e-05  1.7e-05
 5.3e-06  2.8e-05  4.8e-06  8.4e-06   ...   0.00015  1.6e-05  0.49022  2.6e-05
 4.8e-06  5.6e-06 1.06e-05  1.5e-05   ...   6.3e-06  1.3e-05 2.68e-05  0.50255
xi: float32 size=6400x6400

0.08672 0.09111 0.07268 0.10268   ...  0.06562 0.0652 0.09805 0.09459
err: float32 size=6400

-0.22102 -0.2293 -0.2118 -0.2582   ...  -0.2056 -0.2106 -0.2412 -0.243
coerr: float32 size=6400

None
rho: object

0.0001388192177
delta: float64

1 1 1 1   ...  1 1 1 1
theta: float32 size=6400

0.90006 0.90004 0.90002 0.89994   ...  0.89998 0.89999 0.89996 0.89994
gamma: float32 size=6400

By default, all the data is pretty-printed to fit your current terminal column width, with a narrow field width, pytorch-style. But the --noabbrev and --nometa flags gets rid of pretty-printing and metadata to produce an awk-friendly format for processing.

Other flags provide a swiss-army knife array of slicing and summarization options, to make it a useful tool for giving a quick view of what is happening in your data files. What is the mean and variance and L-infinity norm of a block of 14 numbers in the middle of my matrix?

> npycat params_001.npz --key=xi --slice=[25:27,3:10] --mean --std --linf
 4.91e-06    0.0001   4.9e-06  1.09e-05  1.93e-05  0.000118  1.01e-05
 0.000318  2.42e-05  0.000182   9.1e-06  1.88e-05  4.02e-05   0.00011
float32 size=2x7 mean=0.000069 std=0.000087 linf=0.000318

Is that theta vector really all 6400 ones from beginning to end?

> npycat params_000.npz --key=theta --min --max
1 1 1 1   ...  1 1 1 1
float32 size=6400 max=1.000000 min=1.000000

Also npycat is smart about using memory mapping when possible so that the start and end of huge arrays can be printed quickly without bringing the whole contents of an enormous file into memory first. It is fast.

The full usage page:

npycat --help
usage: npycat [-h] [--slice slice] [--unpackbits [axis]] [--key key] [--shape]
              [--type] [--mean] [--std] [--var] [--min] [--max] [--l0] [--l1]
              [--l2] [--linf] [--meta] [--data] [--abbrev] [--name] [--kname]
              [--raise]
              [file [file ...]]

prints the contents of numpy .npy or .npz files.

positional arguments:
  file                 filenames with optional slices such as file.npy[:,0]

optional arguments:
  -h, --help           show this help message and exit
  --slice slice        slice to apply to all files
  --unpackbits [axis]  unpack single-bits from byte array
  --key key            key to dereference in npz dictionary
  --shape              show array shape
  --type               show array data type
  --mean               compute mean
  --std                compute stdev
  --var                compute variance
  --min                compute min
  --max                compute max
  --l0                 compute L0 norm, number of nonzeros
  --l1                 compute L1 norm, sum of absolute values
  --l2                 compute L2 norm, euclidean size
  --linf               compute L-infinity norm, max absolute value
  --meta               use --nometa to suppress metadata
  --data               use --nodata to suppress data
  --abbrev             use --noabbrev to suppress abbreviation of data
  --name               show filename with metadata
  --kname              show key name from npz dictionaries
  --raise              raise errors instead of catching them

examples:
  just print the metadata (shape and type) for data.npy
    npycat data.npy --nodata

  show every number, and the mean and variance, in a 1-d slice of a 5-d tensor
    npycat tensor.npy[0,0,:,0,1] --noabbrev --mean --var

Posted by David at 08:57 AM | Comments (0)

December 18, 2017

In Code We Trust?

As world leaders show themselves prone to falsehood, corruption, greed, and malice, it is tempting to find a new authority in which to place our trust. In today's NYT, Tim Wu observes that rise of Bitcoin evidences humanity's new trust in code: "In our fear of human error, we are putting an increasingly deep faith in technology."

But is this faith well-placed if we do not know how code works or why it does what it does?

Trust in AI Today is about Trust in Testing

Take AI systems. Deep networks used to parse speech or recognize images are subject to massive batteries of tests before they are used. And so in that sense they are more scrutinized than any human person we might hire to do the same job. Trusting a highly scrutinized system seems much better than trusting something untested.

But here is one way that modern AI falls short: we do not expect most AIs to justify, explain, or account for their thinking. And perhaps we do not feel the need for any explanation. Even though explainability is often brought up in the context of medical decisions, my physician friends live in a world of clinical trials, and many of them believe that such rigorous testing on its own is the ultimate proof of utility. You can have all the theories in the world about why something should work, but no theory is as important as experimental evidence of utility. What other proof do we need beyond a rigorous test? Who cares what anybody claims about why it should work, as long as it actually does?

Battle: LeCun vs Rahimi

How much faith to place in empirical versus theoretical results is a debate that is currently raging among AI researchers. On the sidelines of the NIPS 2017 conference, a pitched argument broke out between Yann LeCun (the empiricist) and Ali Rahimi (the theoretician), who disagree on whether empirical AI results without a theoretical foundation just amount to a modern form of alchemy.

I side with Rahimi in revulsion against blind empiricism, but maybe I have different reasons than he. I do not worship the mathematics of rigorous theory. I think the relationship with humans is what is important. We should not trust code unless a person is able to understand some human-interpretable rules that govern its behavior.

The Mathematics of Interpretability

There are two reasons that test results need to be complemented by understandable rules. One is mathematical, and the other is philosophical.

Math first. Our modern AI systems, by their nature, respond to thousands of bits of input. So we should hold any claim of thoroughness of testing up against the harsh fact that visiting each of 2^1000 possible input possibilities - just a few hundred bytes of distinctive input state - would require more tests than atoms in the observable universe, even if every atom had a whole extra universe within it. Most realistic input spaces are far larger, and therefore no test can be thorough in the sense of testing any significant portion of the possibilities.

Furthermore, a sample can only accurately summarize a distribution under the assumption that the world never changes. But humanity imposes a huge rate of change on the world: we change our climate rapidly, we disrupt our governments and businesses regularly, we change our technology faster, and whenever we create a new computer system, adversaries immediately try to change the rules to try to beat it.

Testing is helpful, but "exhaustive" testing is an illusion.

The Philosophy of Interpretability

Philosophy next. This impossibility of testing every possible situation in advance is not a new problem: it has been faced by humanity forever (and, arguably, it is also one of the core problems facing all biological life).

It is in response to this state explosion that mankind invented philosophy, law, engineering, and science. These assemblages of rules are an attempt to distill what we think is important about the individual outcomes we have observed, so that when unanticipated situations arise, we can turn to our old rules and make good, sound decisions again. That is the purpose of ethics, case law, and construction standards. That is the reason that the scientific method is not just about observations, but about creating models and hypotheses before making observations.

We should hold our code to the same standard. It is not good enough for it to perform well on a test. Code should also follow a set of understandable rules that can anticipate its behavior.

Humans need interpretable rules so that we can play our proper role in society. We are the deciders. And to decide about using a machine, we need to be able to see whether the model of action used by the machine matches up with what we think it should be doing, so that when it inevitably faces the many situations in a changing world that will have never been tested before, we can still anticipate its behavior.

If the world never changes and the state space is small, mechanisms are not so important: tests are enough. But that is not the purpose of code in the modern world. Code is humanity's way of putting complexity in a bottle. Therefore its use demands explanations.

Are Understandable Models Possible?

This philosophy of rule-making all sounds vague. Is it possible to do this usefully while still creating the magic intelligent success of deep networks?

I am sure that it is possible, although I don't think it is necessarily easy. There is potentially a bit of math involved. And the world of AI may be easier to explain, or it may not. But it is worth a try.

So, I think, that will be the topic of my dissertation!


Posted by David at 10:00 PM | Comments (0)

December 14, 2017

Net Kleptocracy

Dear A.G. Schneiderman,

My address and my wife's name was fraudulently used in a public comment filed in support of today's horrible FCC vote to repeal net neutrality protections.

The fraud is particularly infuriating because, as readers of this blog know, I was one of the engineers who devoted two decades of my life to building fundamental Internet technologies....

Continue reading "Net Kleptocracy"
Posted by David at 09:37 PM | Comments (1)

November 30, 2017

It's Our Responsibility

It is repulsive how Trump's daily actions transform American power into a force for evil. But we cannot turn away. In the end, it is our country, our system, and our choice. There is no more democratic nation, none with more freedom of speech, none with more vigorous public debate, none with deeper civic institutions. We cannot blame the evil on some tyrant or some invader. We debated, we campaigned, we voted. Trump is the one we chose.

His failure is our failure. It is our responsibility.

The ongoing conversation about Trump's obvious shortcomings misses the point. It is not about him. It is about us. We need to figure out how to remedy our failure as a democracy.


Posted by David at 09:39 AM | Comments (0)

Calendar
November 2021
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30
Projects
Search

Recent Entries
Assistant Professor at NEU Khoury
PhD Defense
Antivax, Antiglyphosate
Global Catastrophizing
Passwords should be illegal
Deception is a Bug
Rewriting a Deep Generative Model
David's Tips on How to Read Pytorch
A COVID Battle Map
COVID-19 Chart API
The Beginning
No Testing is not Cause for Optimism
Two Views of the COVID-19 Crisis
The Purpose of AI
npycat for npy and npz files
In Code We Trust?
Net Kleptocracy
It's Our Responsibility
Volo Ergo Sum
A Crisis of Purpose
Reinvention
Government is Not the Problem
Oriental Exclusion
David Hong-Toh Bau, Sr
Dear Senator Collins
Trump is a Two-Bit Dictator
Network Dissection
Learnable Programming
Beware the Index Fund
Does Watching Fox News Kill You?
Our National Identity
Outrage is Not Enough
A Warning From 1937
Nativist?
A Demon-Haunted World
By the People, For the People
Integrity in Government
Thinking Slow
Whose Country?
Starting at MIT
When to Sell
One-Off Depreciation
Confidence Games
Making a $400 Linux Laptop
Teaching About Data
Code Gym
Musical.js
Pencil Code at Worcester Technical High School
A Bad Chrome Bug
PhantomJS and Node.JS
Integration Testing in Node.js
Second Edition of Pencil Code
Learning to Program with CoffeeScript
Teaching Math Through Pencil Code
Hour of Code at Lincoln
Hour of Code at AMSA
A New Book and a Thanksgiving Wish
Pencil Code: Lesson on Angles
Pencil Code: Lesson on Lines
Pencil Code: a First Look
CoffeeScript Syntax for Kids
CSS Color Names
For Versus Repeat
Book Sample Page
Teaching Programming and Defending the Middle Class
TurtleBits at Beaver Country Day
Book Writing Progress
Lessons from Kids
Await and Defer
Ticks, Animation, and Queueing in TurtleBits
Using the TurtleBits Editor
Starting with Turtlebits
Turtle Bits
No Threshold, No Limit
Local Variable Debugging with see.js
Mapping the Earth with Complex Numbers
Conformal Map Viewer
Jobs in 1983
The Problem With China
Omega Improved
Made In America Again
Avoiding Selectors for Beginners
Turtle Graphics Fern with jQuery
Learning To Program with jQuery
jQuery-turtle
Python Templating with @stringfunction
PUT and DELETE in call.jsonlib.com
Party like it's 1789
Using goo.gl with jsonlib
Simple Cross-Domain Javascript HTTP with call.jsonlib.com
Dabbler Under Version Control
Snowpocalypse Hits Boston
Heidi's Sudoku Hintpad
Social Responsibility in Tech
The First Permanent Language
A New Framework For Finance
Box2D Web
Lincoln School Construction
Stuck Pixel Utility
Fixing the Deficit
Archives
Links
Older Writing
About