Artificial Intelligence - History And Timeline

 

1942

The Three Laws of Robotics by science fiction author Isaac Asimov occur in the short tale "Runaround."

1943

Emil Post, a mathematician, talks about "production systems," a notion he adopted for the 1957 General Problem Solver.

1943

"A Logical Calculus of the Ideas of Immanent in Nervous Activity," a study by Warren McCulloch and Walter Pitts on a computational theory of neural networks, is published.

1944

The Teleological Society was founded by John von Neumann, Norbert Wiener, Warren McCulloch, Walter Pitts, and Howard Aiken to explore, among other things, nervous system communication and control.

1945

In his book How to Solve It, George Polya emphasizes the importance of heuristic thinking in issue solving.

1946

In New York City, the first of eleven Macy Conferences on Cybernetics gets underway. "Feedback Mechanisms and Circular Causal Systems in Biological and Social Systems" is the focus of the inaugural conference.

1948

Norbert Wiener, a mathematician, publishes Cybernetics, or Control and Communication in the Animal and the Machine.

1949

In his book The Organization of Behavior, psychologist Donald Hebb provides a theory for brain adaptation in human education: "neurons that fire together connect together."

1949

Edmund Berkeley's book Giant Brains, or Machines That Think, is published.

1950

Alan Turing's "Computing Machinery and Intelligence" describes the Turing Test, which attributes intelligence to any computer capable of demonstrating intelligent behavior comparable to that of a person.

1950

Claude Shannon releases "Programming a Computer for Playing Chess," a groundbreaking technical study that shares search methods and strategies.

1951

Marvin Minsky, a math student, and Dean Edmonds, a physics student, create an electronic rat that can learn to navigate a labyrinth using Hebbian theory.

1951

John von Neumann, a mathematician, releases "General and Logical Theory of Automata," which reduces the human brain and central nervous system to a computer.

1951

For the University of Manchester's Ferranti Mark 1 computer, Christopher Strachey produces a checkers software and Dietrich Prinz creates a chess routine.

1952

Cyberneticist W. Edwards wrote Design for a Brain: The Origin of Adaptive Behavior, a book on the logical underpinnings of human brain function. Ross Ashby is a British actor.

1952

At Cornell University Medical College, physiologist James Hardy and physician Martin Lipkin begin developing a McBee punched card system for mechanical diagnosis of patients.

1954

Science-Fiction Thinking Machines: Robots, Androids, Computers, edited by Groff Conklin, is a theme-based anthology.

1954

The Georgetown-IBM project exemplifies the power of text machine translation.

1955

Under the direction of economist Herbert Simon and graduate student Allen Newell, artificial intelligence research began at Carnegie Tech (now Carnegie Mellon University).

1955

For Scientific American, mathematician John Kemeny wrote "Man as a Machine."

1955

In a Rockefeller Foundation proposal for a Dartmouth University meeting, mathematician John McCarthy coined the phrase "artificial intelligence."

1956

Allen Newell, Herbert Simon, and Cliff Shaw created Logic Theorist, an artificial intelligence computer software for proving theorems in Alfred North Whitehead and Bertrand Russell's Principia Mathematica.

1956

The "Constitutional Convention of AI," a Dartmouth Summer Research Project, brings together specialists in cybernetics, automata, information theory, operations research, and game theory.

1956

On television, electrical engineer Arthur Samuel shows off his checkers-playing AI software.

1957

Allen Newell and Herbert Simon created the General Problem Solver AI software.

1957

The Rockefeller Medical Electronics Center shows how an RCA Bizmac computer application might help doctors distinguish between blood disorders.

1958

The Computer and the Brain, an unfinished work by John von Neumann, is published.

1958

At the "Mechanisation of Thought Processes" symposium at the UK's Teddington National Physical Laboratory, Firmin Nash delivers the Group Symbol Associator its first public demonstration.

1958

For linear data categorization, Frank Rosenblatt develops the single layer perceptron, which includes a neural network and supervised learning algorithm.

1958

The high-level programming language LISP is specified by John McCarthy of the Massachusetts Institute of Technology (MIT) for AI research.

1959

"The Reasoning Foundations of Medical Diagnosis," written by physicist Robert Ledley and radiologist Lee Lusted, presents Bayesian inference and symbolic logic to medical difficulties.

1959

At MIT, John McCarthy and Marvin Minsky create the Artificial Intelligence Laboratory.

1960

James L. Adams, an engineering student, built the Stanford Cart, a remote control vehicle with a television camera.

1962

In his short novel "Without a Thought," science fiction and fantasy author Fred Saberhagen develops sentient killing robots known as Berserkers.

1963

John McCarthy developed the Stanford Artificial Intelligence Laboratory (SAIL).

1963

Under Project MAC, the Advanced Research Experiments Agency of the United States Department of Defense began financing artificial intelligence projects at MIT.

1964

Joseph Weizenbaum of MIT created ELIZA, the first software allowing natural language conversation with a computer (a "chatbot").

1965

I am a statistician from the United Kingdom. J. Good's "Speculations Concerning the First Ultraintelligent Machine," which predicts an impending intelligence explosion, is published.

1965

Hubert L. Dreyfus and Stuart E. Dreyfus, philosophers and mathematicians, publish "Alchemy and AI," a study critical of artificial intelligence.

1965

Joshua Lederberg and Edward Feigenbaum founded the Stanford Heuristic Programming Project, which aims to model scientific reasoning and create expert systems.

1965

Donald Michie is the head of Edinburgh University's Department of Machine Intelligence and Perception.

1965

Georg Nees organizes the first generative art exhibition, Computer Graphic, in Stuttgart, West Germany.

1965

With the expert system DENDRAL, computer scientist Edward Feigenbaum starts a ten-year endeavor to automate the chemical analysis of organic molecules.

1966

The Automatic Language Processing Advisory Committee (ALPAC) issues a cautious assessment on machine translation's present status.

1967

On a DEC PDP-6 at MIT, Richard Greenblatt finishes work on Mac Hack, a computer that plays competitive tournament chess.

1967

Waseda University's Ichiro Kato begins work on the WABOT project, which culminates in the unveiling of a full-scale humanoid intelligent robot five years later.

1968

Stanley Kubrick's adaptation of Arthur C. Clarke's science fiction novel 2001: A Space Odyssey, about the artificially intelligent computer HAL 9000, is one of the most influential and highly praised films of all time.

1968

At MIT, Terry Winograd starts work on SHRDLU, a natural language understanding program.

1969

Washington, DC hosts the First International Joint Conference on Artificial Intelligence (IJCAI).

1972

Artist Harold Cohen develops AARON, an artificial intelligence computer that generates paintings.

1972

Ken Colby describes his efforts using the software program PARRY to simulate paranoia.

1972

In What Computers Can't Do, Hubert Dreyfus offers his criticism of artificial intelligence's intellectual basis.

1972

Ted Shortliffe, a doctorate student at Stanford University, has started work on the MYCIN expert system, which is aimed to identify bacterial illnesses and provide treatment alternatives.

1972

The UK Science Research Council releases the Lighthill Report on Artificial Intelligence, which highlights AI technological shortcomings and the challenges of combinatorial explosion.

1972

The Assault on Privacy: Computers, Data Banks, and Dossiers, by Arthur Miller, is an early study on the societal implications of computers.

1972

INTERNIST-I, an internal medicine expert system, is being developed by University of Pittsburgh physician Jack Myers, medical student Randolph Miller, and computer scientist Harry Pople.

1974

Paul Werbos, a social scientist, has completed his dissertation on a backpropagation algorithm that is currently extensively used in artificial neural network training for supervised learning applications.

1974

The memo discusses the notion of a frame, a "remembered framework" that fits reality by "changing detail as appropriate." Marvin Minsky distributes MIT AI Lab document 306 on "A Framework for Representing Knowledge."

1975

The phrase "genetic algorithm" is used by John Holland to explain evolutionary strategies in natural and artificial systems.

1976

In Computer Power and Human Reason, computer scientist Joseph Weizenbaum expresses his mixed feelings on artificial intelligence research.

1978

At Rutgers University, EXPERT, a generic knowledge representation technique for constructing expert systems, becomes live.

1978

Joshua Lederberg, Douglas Brutlag, Edward Feigenbaum, and Bruce Buchanan started the MOLGEN project at Stanford to solve DNA structures generated from segmentation data in molecular genetics research.

1979

Raj Reddy, a computer scientist at Carnegie Mellon University, founded the Robotics Institute.

1979

While working with a robot, the first human is slain.

1979

Hans Moravec rebuilds and equips the Stanford Cart with a stereoscopic vision system after it has evolved into an autonomous rover over almost two decades.

1980

The American Association of Artificial Intelligence (AAAI) holds its first national conference at Stanford University.

1980

In his Chinese Room argument, philosopher John Searle claims that a computer's modeling of action does not establish comprehension, intentionality, or awareness.

1982

Release of Blade Runner, a science fiction picture based on Philip K. Dick's tale Do Androids Dream of Electric Sheep? (1968).

1982

The associative brain network, initially developed by William Little in 1974, is popularized by physicist John Hopfield.

1984

In Fortune Magazine, Tom Alexander writes "Why Computers Can't Outthink the Experts."

1984

At the Microelectronics and Computer Consortium (MCC) in Austin, TX, computer scientist Doug Lenat launches the Cyc project, which aims to create a vast commonsense knowledge base and artificial intelligence architecture.

1984

Orion Pictures releases the first Terminator picture, which features robotic assassins from the future and an AI known as Skynet.

1986

Honda establishes a research facility to build humanoid robots that can cohabit and interact with humans.

1986

Rodney Brooks, an MIT roboticist, describes the subsumption architecture for behavior-based robots.

1986

The Society of Mind is published by Marvin Minsky, who depicts the brain as a collection of collaborating agents.

1989

The MIT Artificial Intelligence Lab's Rodney Brooks and Anita Flynn publish "Fast, Cheap, and Out of Control: A Robot Invasion of the Solar System," a paper discussing the possibility of sending small robots on interplanetary exploration missions.

1993

The Cog interactive robot project is launched at MIT by Rodney Brooks, Lynn Andrea Stein, Cynthia Breazeal, and others.

1995

The phrase "generative music" was used by musician Brian Eno to describe systems that create ever-changing music by modifying parameters over time.

1995

The MQ-1 Predator unmanned aerial aircraft from General Atomics has entered US military and reconnaissance duty.

1997

Under normal tournament settings, IBM's Deep Blue supercomputer overcomes reigning chess champion Garry Kasparov.

1997

In Nagoya, Japan, the inaugural RoboCup, an international tournament featuring over forty teams of robot soccer players, takes place.

1997

NaturallySpeaking is Dragon Systems' first commercial voice recognition software product.

1999

Sony introduces AIBO, a robotic dog, to the general public.

2000

The Advanced Step in Innovative Mobility humanoid robot, ASIMO, is unveiled by Honda.

2001

At Super Bowl XXXV, Visage Corporation unveils the FaceFINDER automatic face-recognition technology.

2002

The Roomba autonomous household vacuum cleaner is released by the iRobot Corporation, which was created by Rodney Brooks, Colin Angle, and Helen Greiner.

2004

In the Mojave Desert near Primm, NV, DARPA hosts its inaugural autonomous vehicle Grand Challenge, but none of the cars complete the 150-mile route.

2005

Under the direction of neurologist Henry Markram, the Swiss Blue Brain Project is formed to imitate the human brain.

2006

Netflix is awarding a $1 million prize to the first programming team to create the greatest recommender system based on prior user ratings.

2007

DARPA has announced the commencement of the Urban Challenge, an autonomous car competition that will test merging, passing, parking, and navigating traffic and junctions.

2009

Under the leadership of Sebastian Thrun, Google launches its self-driving car project (now known as Waymo) in the San Francisco Bay Area.

2009

Fei-Fei Li of Stanford University describes her work on ImageNet, a library of millions of hand-annotated photographs used to teach AIs to recognize the presence or absence of items visually.

2010

Human manipulation of automated trading algorithms causes a "flash collapse" in the US stock market.

2011

Demis Hassabis, Shane Legg, and Mustafa Suleyman developed DeepMind in the United Kingdom to educate AIs how to play and succeed at classic video games.

2011

Watson, IBM's natural language computer system, has beaten Jeopardy! Ken Jennings and Brad Rutter are the champions.

2011

The iPhone 4S comes with Apple's mobile suggestion assistant Siri.

2011

Andrew Ng, a computer scientist, and Google colleagues Jeff Dean and Greg Corrado have launched an informal Google Brain deep learning research cooperation.

2013

The European Union's Human Brain Project aims to better understand how the human brain functions and to duplicate its computing capabilities.

2013

Stop Killer Robots is a campaign launched by Human Rights Watch.

2013

Spike Jonze's science fiction drama Her has been released. A guy and his AI mobile suggestion assistant Samantha fall in love in the film.

2014

Ian Goodfellow and colleagues at the University of Montreal create Generative Adversarial Networks (GANs) for use in deep neural networks, which are beneficial in making realistic fake human photos.

2014

Eugene Goostman, a chatbot that plays a thirteen-year-old kid, is said to have passed a Turing-like test.

2014

According to physicist Stephen Hawking, the development of AI might lead to humanity's extinction.

2015

DeepFace is a deep learning face recognition system that Facebook has released on its social media platform.

2016

In a five-game battle, DeepMind's AlphaGo software beats Lee Sedol, a 9th dan Go player.

2016

Tay, a Microsoft AI chatbot, has been put on Twitter, where users may teach it to send abusive and inappropriate posts.

2017

The Asilomar Meeting on Beneficial AI is hosted by the Future of Life Institute.

2017

Anthony Levandowski, an AI self-driving start-up engineer, formed the Way of the Future church with the goal of creating a superintelligent robot god.

2018

Google has announced Duplex, an AI program that uses natural language to schedule appointments over the phone.

2018

The General Data Protection Regulation (GDPR) and "Ethics Guidelines for Trustworthy AI" are published by the European Union.

2019

A lung cancer screening AI developed by Google AI and Northwestern Medicine in Chicago, IL, surpasses specialized radiologists.

2019

Elon Musk cofounded OpenAI, which generates realistic tales and journalism via artificial intelligence text generation. Because of its ability to spread false news, it was previously judged "too risky" to utilize.

2020

TensorFlow Quantum, an open-source framework for quantum machine learning, was announced by Google AI in conjunction with the University of Waterloo, the "moonshot faculty" X, and Volkswagen.

~ Jai Krishna Ponnappan

Find Jai on Twitter | LinkedIn | Instagram

You may also want to read more about Artificial Intelligence here.

Artificial Intelligence - SVM Or Support Vector Machine For Active Learning or AL - ALSVM.

 

PROLOGUE.

With the growing need for multimedia information retrieval from the Web, such as picture and video retrieval, it's becoming more difficult to train a classifier when the training dataset contains a small number of labelled data and a big number of unlabeled data. 

Traditional supervised and unsupervised learning approaches are ineffective in solving such issues, especially when the data is in a high-dimensional environment. 

Many strategies have been presented in recent years that may be essentially split into two categories: semi-supervised and active learning (AL). 

Since the turn of the century, a number of academics have introduced Active Learning with SVM (ALSVM) methods, which have been regarded as an efficient technique for dealing with high-dimensionality issues. 

We discuss the state-of-the-art of ALSVM for tackling classification issues in this work, given their fast progress. 

 

CONTEXT.

The overall structure of AL is shown below.

 

It is evident that the learner may enhance the classifier by actively picking the "optimal" data from the possible query set Q and adding it to the current labeled training set L after receiving its label throughout the procedures. 

AL's most important feature is its sample selection criteria. 

AL was formerly mostly utilized in conjunction with the neural network algorithm and other learning algorithms. 

The sample minimizing either variance (D.  A. Cohn, Ghahramani, & Jordan, 1996), bias (D. A. Cohn, 1997), or generalization error (Roy & McCallum, 2001) is queried to the oracle in statistical AL. 

Although these approaches have a solid theoretical basis, they are limited in their application due to two frequent issues: how to estimate the posterior distribution of the data and the excessively high computational cost. 

To address the aforementioned two issues, a number of version space-based AL methods have been proposed, which are based on the assumption that the target function can be perfectly expressed by one hypothesis in the version space and that the sample that can reduce the volume of the version space is chosen. 

Query by committee (Freund, Seung, Shamir, & Tishby, 1997) and SG AL are two examples (D. Cohn, Atlas, & Ladner, 1994). 

However, before the advent of version space-based ALSVMs, they were intractable due to the complexity of version space. 

Researchers have combined AL with SVM to cope with semi-supervised learning issues, such as distance-based (Tong & Koller, 2001), RETIN (Gosselin & Cord, 2004), and Multi-view (Cheng & Wang, 2007) based ALSVMs, thanks to the popularity of SVM in the 1990s. 

VERSION SPACE BASED ACTIVE LEARNING WITH SVM

Almost all extant heuristic ALSVMs aim to discover the sample that can minimize the size of the version space, either explicitly or indirectly. 

In this part, we'll go through their theoretical underpinning before going over several common ALSVMs. 

Existing well-known ALSVMs are discussed in the context of version space theory, followed by a short description of certain mixed techniques. 

Finally, we'll talk about ALSVM research trends and draw conclusions for the study. 

Theory of Version Space.

Based on the Approximation of Probability Machine learning's purpose is to create a consistent classifier with the lowest generalization error constraint. 

The Gibbs generalization error bound (McAllester, 1998) is defined as follows: 

where PH is a prior distribution over hypothesis space H, V(z) is the version space of the training set z, m is the number of z, and d is a constant in the range [0, 1]. 

If the distribution of the version space is uniform, the volume of the version space controls the generalization error limits of the consistent classifiers. 

This gives a theoretical foundation for ALSVMs based on version space. 

 

With SVM, query by committee.

(Freund et al., 1997) developed a technique in which 2k classifiers were randomly picked, and the sample on which these classifiers have the most disagreement may roughly half the version space, and then the oracle was queried. 

The intricacy of the structure of the version space, on the other hand, makes random sampling within it challenging. 

(Warmuth, Ratsch, Mathieson, Liao, & Lemmem, 2003) successfully used a billiards approach to randomly choose classifiers in the SVM version space, and testing revealed that its performance was equivalent to that of the standard distance-based ALSVM (SD-ALSVM), which will be discussed later. 

The flaw is that the procedures take a long time to complete. 

Standard Active Learning with SVM over a Long Distance.

The version space for SVM is defined as follows: 

where Φ(.) denotes the function which map the original input space X into a high-dimensional space Φ X )( , and W denotes the parameter space. SVM has two properties which lead to its tractability with AL. 

The first is its duality property that each point w in V corresponds to one hyperplane in Φ X )( which divides Φ X )( into two parts and vice versa. 

The other property is that the solution of SVM w* is the center of the version space when the version space is symmetric or near to its center when it is asymmetric.

(Tong & Koller, 2001) deduced a lemma from the above two properties: the sample closest to the decision boundary may reduce the predicted size of the version space the quickest. 

As a result, the oracle will be requested for the sample closest to the decision border shown in the Figure below: 

This is the SD-ALSVM, which has minimal extra calculations for picking the query sample and excellent performance in real-world applications. 

Distance Based Active Learning with SVM in Batch Running Mode 

(Tong & Koller, 2001) used batch query to choose several samples that are closest to the decision boundary. 

However, as shown in figure below, adding a batch of such samples does not guarantee t
he maximum decrease in the size of version space. 

Although each sample can almost half the version space, three samples combined can only lower the version space by around 1/2, less than 7/8. 

This was attributed to the tiny angles between their induced hyperplanes, as can be shown. 

(Brinker, 2003) developed a novel selection technique that incorporates a diversity measure that examines the angles between the induced hyperplanes to solve this issue. 

In the present round, if the labeled set is L and the pool query set is Q, then the further added sample xq should be independent on the frontier and an adaptive approach was devised to adjust s throughout the feedback rounds based on the diversity requirement. 

Where yxkyxh)() denotes the number of relevant and irrelevant samples in the queried set in the ith iteration, and rrel(i) and rirrl(i) signify the number of relevant and irrelevant samples in the queried set in the ith iteration. 

The amount of relevant and irrelevant samples in the searched set will be about equal as a result of this method. 

The Mean Version Space Criterion.

(He, Li, Zhang, Tong, & Zhang, 2004) suggested a selection criteria based on reducing the mean version space, which is described as

where )(( ki xVVol + ( )(( ki xVVol − ) denotes the volume of the version space after adding an unlabelled sample xk into the ith round training set. 

The volume of the version space as well as the posterior probability are included in the mean version space. 

As a result, they concluded that the criteria is superior than the SD-ALSVM. 

However, this method's calculation takes a long time. 

Active Learning with Multiple Views.

Multi-view approaches, in contrast to algorithms that are based on a single feature set, are based on numerous sub-feature sets. 

Several classifiers are trained on various sub-feature sets initially. 

The contention set, from which questioned examples are chosen, is made up of the samples on which the classifiers have the most disagreements. 

First, it was used in AL (I. Muslea, Minton, & Knoblock, 2000), and then it was combined with ALSVM (Cheng & Wang, 2007) to generate a CoSVM algorithm that outperformed the SD-ALSVM. 

Because they see the data from diverse perspectives, several classifiers may detect the unusual examples. 

This attribute is quite beneficial for locating components that belong to the same category. 

Multi-view based approaches, on the other hand, need that the relevant classifier be able to correctly categorize the samples and that all feature sets be uncorrelated. 

In real-world applications, ensuring this requirement is tough. 

MIXED ACTIVE LEARNING.

In this part, instead of single AL techniques as in the previous sections, we'll talk about two mixed AL modes: one that combines diverse selection criteria, and the other that incorporates semi-supervised learning into AL. 

Hybrid Active Learning.

Rather than creating a new AL algorithm that works in all scenarios, some academics think that integrating several approaches, which are generally complimentary, is a superior approach since each method has its own set of benefits and drawbacks. 

The parallel mode structure of the hybrid approach is obvious. 

The important thing to remember here is how to adjust the weights of various AL techniques. 

Most previous methods employed the simplest method of setting fixed weights based on experience. 

Most Relevant/Irrelevant (L. Zhang, Lin, & Zhang, 2001) techniques may assist to stabilize the decision border, but they have poor learning rates; traditional distance-based approaches have high learning rates, but their frontiers are unstable at the early feedbacks. 

Taking this into account, (Xu, Xu, Yu, & Tresp, 2003) combined these two tactics to get greater results than if they had used just one. 

As previously mentioned, diversity and distance-based techniques are complimentary, and (Brinker, 2003), (Ferecatu, Crucianu, & Boujemaa, 2004), and (Dagli, Rajaram, & Huang, 2006), respectively, coupled angle, inner product, and entropy diversity strategies with conventional distance-based strategies. 

However, the fixed weights technique does not work well with all datasets or learning iterations. 

As a result, the weights should be adjusted dynamically. 

All the weights were started with the same value in (Baram, El-Yaniv, & Luz, 2004), and were adjusted in subsequent iterations using the EXP4 method. 

As a consequence, the resultant AL algorithm is routinely demonstrated to perform almost as good as, and occasionally even better than, the best algorithm in the ensemble. 

Semi-Supervised Active Learning.

1. Active Learning with Transductive SVM (Semi-Supervised Active Learning) 

A few labeled data in the early stages of SD-ALSVM may cause a significant departure of the current solution from the genuine solution; however, if unlabeled samples are taken into account, the solution may be closer to the true solution. 

(Wang, Chan, & Zhang, 2003) shown that the smaller the version space is, the closer the present answer is to the genuine one. 

To create more accurate intermediate solutions, they used Transductive SVM (TSVM). 

However, some research (T. Zhang & Oles, 2000) questioned whether TSVM is as useful in principle as it is in reality when dealing with unlabeled data. 

Instead, (Hoi & Lyu, 2005) used semi-supervised learning approaches based on Gaussian fields and Harmonic functions, with considerable gains observed. 

2. Incorporating EM into Active Learning  is the second step. 

Expectation Maximization (EM) was integrated with the approach of querying by committee (McCallum & Nigam, 1998). 

And (Ion Muslea, Minton, & Knoblock, 2002) combined the Multi-view AL method with EM to create the Co-EMT algorithm, which may be used when the views are incompatible and correlated. 

PROJECTIONS FOR THE FUTURE.

 

How to Begin the Active Learning Process.

Because AL may be thought of as the issue of finding a target function in version space, a solid starting classifier is crucial. 

When the objective category is large, the initial classifier becomes more significant since a defective one might lead to convergence to a local optimum solution, which means that certain areas of the objective category may not be accurately covered by the final classifier. 

Long-term learning (Yin, Bhanu, Chang, & Dong, 2005), and pre-cluster (Engelbrecht & BRITS, 2002) techniques are all promising. 

Feature-Based Active Learning

In AL, the oracle's comments may also aid in identifying essential characteristics, and (Raghavan, Madani, & Jones, 2006) shown that such efforts can greatly increase the final classifier's performance. 

Principal Components Analysis was used to discover essential aspects in (Su, Li, & Zhang, 2001). 

There are few reports on the subject that we are aware of. 

 

Active Learning Scaling

The scalability of AL to extremely big databases has not yet been thoroughly investigated. 

However, it is a significant problem in a number of real-world applications. 

Some ways to indexing databases (Lai, Goh, & Chang, 2004) and overcoming the concept complications associated with the dataset's scalability have been offered (Panda, Goh, & Chang, 2006). 

 

INFERENCE

In this study, we review the ALSVM approaches, which have been the subject of intensive research since 2000. 

Within the context of the notion of version space minimization, we initially concentrate on descriptions of heuristic ALSVM techniques. 

Then, to balance the shortcomings of single approaches, mixed methods are proposed, and lastly, future research trends concentrate on strategies for choosing the initial labeled training set, feature-based AL, and AL scaling to extremely large databases.

GLOSSARY OF TERMS USED:

Heuristic Active Learning is a class of active learning algorithms in which the sample selection criteria are determined by a heuristic objective function. Version space based active learning, for example, selects a sample that reduces the size of the version space. 

Hypothesis Space: The collection of all hypotheses in which the objective hypothesis is expected to be found is known as Hypothesis Space. 

Semi-Supervised Learning: A type of learning methods in which the classifier is trained using both labelled and unlabelled data from the training dataset. 

Statistical Active Learning: A class of active learning algorithms in which the sample selection criteria are based on statistical objective functions such minimization of generalization error, bias, and variance. In most cases, statistical active learning is statistically optimum. 

Supervised Learning: The collection of learning algorithms in which all of the samples in the training dataset are labeled is known as supervised learning. 

Unsupervised Learning is a class of learning algorithms in which the training dataset samples are entirely unlabeled. 

Version Space: The subset of the hypothesis space that is compatible with the training set is known as version space.

~ Jai Krishna Ponnappan

Find Jai on Twitter | LinkedIn | Instagram

You may also want to read more about Artificial Intelligence here.

REFERENCES AND FURTHER READING:

Baram, Y., El-Yaniv, R., & Luz, K. (2004). Online Choice of Active Learning Algorithms. Journal of 

Machine Learning Research, 5, 255-291.

Brinker, K. (2003). Incorporating Diversity in Active Learning with Support Vector Machines. Paper 

presented at the International Conference on Machine Learning.

Cheng, J., & Wang, K. (2007). Active learning for image retrieval with Co-SVM. Pattern Recognition, 

40(1), 330-334

Cohn, D., Atlas, L., & Ladner, R. (1994). Improving Generalization with Active Learning. Machine Learning, 15, 201-221.

Cohn, D. A. (1997). Minimizing Statistical Bias with Queries. In Advances in Neural Information Processing Systems 9, Also appears as AI Lab Memo 1552, CBCL Paper 124. M. Mozer et al, eds.

Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996). Active Learning with Statistical Models. Journal of Artificial Intelligence Research, 4, 129-145.

Cord, M., Gosselin, P. H., & Philipp-Foliguet, S. (2007). Stochastic exploration and active learning for image retrieval. Image and Vision Computing, 25(1), 14-23.

Dagli, C. K., Rajaram, S., & Huang, T. S. (2006). Utiliz￾ing Information Theoretic Theoretic Diversity for SVM Active Learning. Paper presented at the International Conference on Pattern Recognition, Hong Kong.

Engelbrecht, A. P., & BRITS, R. (2002). Supervised Training Using an Unsuerpvised Approach to Active Learning. Neural Processing Letters, 15, 14.

Ferecatu, M., Crucianu, M., & Boujemaa, N. (2004). Reducing the redundancy in the selection of samples for SVM-based relevance feedback 

Freund, Y., Seung, H. S., Shamir, E., & Tishby, N. (1997). Selective Sampling Using the Query by Committee Algorithm. Machine Learning, 28, 133-168.

Gosselin, P. H., & Cord, M. (2004). RETIN AL: an active learning strategy for image category retrieval.Paper presented at the International Conference on Image Processing.

He, J., Li, M., Zhang, H.-J., Tong, H., & Zhang, C. (2004). Mean version space: a new active learning 

method for content-based image retrieval. Paper presented at the International Multimedia Conference 

Proceedings of the 6th ACM SIGMM International Workshop on Mulitimedia Information Retrieval.

Hoi, S. C. H., & Lyu, M. R. (2005). A semi-supervised active learning framework for image retrieval. Paper presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

Lai, W.-C., Goh, K., & Chang, E. Y. (2004, June). On Scalability of Active Learning for Formulating Query Concepts (long version of the ICME invited paper). Paper presented at the Workshop on Computer Vision Meets Databases (CVDB) in cooperation with ACM International Conference on Management of Data (SIGMOD), Paris.

McAllester, D. A. (1998). Some PAC Bayesian Theorems. Paper presented at the Proceedings of the 11th Annual Conference on Computational Learning Theory, Madison, Wisconsin.

McCallum, A. K., & Nigam, K. (1998). Employing EM and Pool-Based Active Learning for Text Classification. Paper presented at the Proceedings of 15th International Conference on Machine Learning.

Muslea, I., Minton, S., & Knoblock, C. A. (2000). Selective Sampling with Redundant Views. Paper presented at the Proceedings of the 17th National Conference on Artificial Intelligence.

Muslea, I., Minton, S., & Knoblock, C. A. (2002). Active+Semi-Supervised Learning = Robust Multi-View Learning. Paper presented at the Proceedings of the 19th International Conference on Machine Learning.

Panda, N., Goh, K., & Chang, E. Y. (2006). Active Learning in Very Large Image Databases Journal of Multimedia Tools and Applications Special Issue on Computer Vision Meets Databases.

Raghavan, H., Madani, O., & Jones, R. (2006). Active Learning with Feedback on Both Features and 

Instances. Journal of Machine Learning Research, 7, 1655-1686.

Roy, N., & McCallum, A. (2001). Toward Optimal Active Learning Through Sampling Estimation of Error Reduction. Paper presented at the Proceedings of 18th International Conference on Machine Learning.

Su, Z., Li, S., & Zhang, H. (2001). Extraction of Feature Subspaces for Content-based Retrieval Using Relevance Feedback. Paper presented at the ACM Multimedia, Ottawa, Ontario, Canada.

Tong, S., & Koller, D. (2001). Support Vector Machine Active Learning with Application to Text Classification. Journal of Machine Learning Research, 45-66.

Wang, L., Chan, K. L., & Zhang, Z. (2003). Bootstrapping SVM active learning by incorporating unlabelled images for image retrieval. Paper presented at the Proceeding of IEEE Computer Vision and Pattern 

Warmuth, M. K., Ratsch, G., Mathieson, M., Liao, J., & Lemmem, C. (2003). Active Learning in the Drug Discovery Process. Journal of Chemical Information Sciences, 43(2), 667-673.

Xu, Z., Xu, X., Yu, K., & Tresp, V. (2003). A Hybrid Relevance-feedback Approach to Text Retrieval. Paper presented at the Proceedings of the 25th European Conference on Information Retrieval Research, Lecture Notes in Computer Science.

Yin, P., Bhanu, B., Chang, K., & Dong, A. (2005). Integrating Relevance Feedback Techniques for Image Retrieval Using Reinforcement Learning IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1536-1551.

Zhang, L., Lin, F., & Zhang, B. (2001). Support Vector Machine Learning for Image Retrieval. Paper presented at the International Conference on Image Processing.

Zhang, T., & Oles, F. (2000). A Probability Analysis on The Value of Unlabeled Data for Classification Problems. Paper presented at the Proceeding of 17th International Conference of Machine Learning, San Francisco, CA.