by Mark Davidson
In Part 1 of this series we explored the two competing camps of Artificial Intelligence. On the one hand those who believed that intelligence can be described through rules and recall - the Symbolists. On the other hand those who looked at modeling intelligence with networks of connected neurons - the Connectionists. Both of these schools of thought started in the 1950s and developed during the 60s. In many cases they were competing for the same sources of funding.
The mathematical model of a neuron, the work of the Connectionists, was called a perceptron which acted as a classifier to take an input value and transform it into a discrete result. In a pivotal moment during the fragile ascendance of AI, a couple of Symbolists Marvin Minsky and Seymour Papert wrote the book Perceptrons (1969) which applied mathematical rigor to the theory of perceptrons. Minsky and Papert could mathematically prove that a single layer perceptron could not solve some fundamental computer science predicates such as the exclusive or (XOR). They further conjectured that multi-layered perceptrons could not solve general problems. This turned out to be wrong but the damage was done and the book killed neural net research for an entire generation. This gave wiggle room for the emergent field of symbolic AI.
Another key moment in the AI winter occurred In 1974 when Sir James Lighthill issued a report to Parliament on the state of AI research in the UK and it criticized the utter failure of AI to achieve its “grandiose objectives”. He concluded that nothing being done in AI couldn't be done in other sciences. He implied that many of AI's most successful algorithms would grind to a halt on real world problems and were only suitable for solving "toy" versions. This report led to the complete dismantling of AI research in England. Meanwhile, the US did its part when DARPA stopped funding open ended AI research in favor of “mission-oriented direct research”. The AI winter had started and would last until the 1980s.
AI research still happened during this time and academic progress combined with faster systems were leading to some positive results. Expert systems was a form of symbolic AI - which aimed to emulate the decision making ability of a human expert. Expert systems have two essential components: a knowledge base which represents the facts of the world and an inference engine (or rule engine) that evaluates new data against the knowledge base to produce a set of outcomes or recommendations. Essentially, the application of the rule engine to the knowledge base will connect symbols in a relationship similar to an If-Then statement. The challenge was to build the knowledge base and the rule engine from interviewing experts in the field and codifying their experience. They provided useful solutions to some narrow problem domains and it led to a boom in the 80s as corporations attempted to apply these techniques to their areas of focus. The rise of expert systems put an end to the AI winter of the 70s.
In the 80s and 90s, there were a number of commercial successes with this approach. One of the biggest success stories of the expert systems era was applied to the field of logic gate design in the VAX 9000. The Synthesis of Integral Design (SID) had over 1000 hand written rules but expanded to find about 384,000 low level rules and could produce logic gate designs much better and more efficiently than designing these gates by hand.
I started my software development career in the early 1990s building the user interface of a rule editor of an expert system which would do vibration analysis of rotating machinery. I had been reading about AI for a few years up until then but was excited to work on a practical application.
There was a lot of hype and inflated expectations as to what expert systems could achieve which led to a lot of investments and new companies formed. Eventually, the promise was underwhelming compared to the hype. The bubble burst and the rise of the Internet led to another AI winter and skepticism about expert systems. They are still around today but these techniques are integrated into existing platforms by SAP, Siebel and Oracle as “business logic”. The lessons of expert systems are applied by productOps in the development of platforms for our clients today. In Part 3 of this series, we see how limitations of the symbolic approach to AI along with the rise of cheap storage, fast computation and faster networks like the Internet gave rise to Deep Learning.