Is research on LLM AI slowing down?

John McCarthy

It looks like the fate of the field of computer technology, wrongly called artificial intelligence, is to alternate between excessive optimism and unbridled pessimism. Here is a sketch of the history of this technology:

1. At the Dartmouth College summer school in 1956, the name artificial intelligence was proposed for computer programs that could perform tasks that had traditionally been considered exclusively human, such as playing chess and translating from one human language to another. The attendees, led by John McCarthy, predicted that within ten years these two problems would be solved. They hoped that by 1966 there would be programs capable of beating the world chess champion, and others that could translate perfectly between any two human languages. When these objectives were not achieved so early, research into artificial intelligence stopped. At universities, research topics in this field were frowned upon, because they were thought to have no future.

The kingdom of lies

Abraham Lincoln

 Abraham Lincoln is credited with having said this:

You can deceive some people all the time. You can deceive everybody part of the time. But you can't deceive everybody all the time.

When I was young, if a politician was caught lying, he must resign, because he had deceived the people. This was true back then. Famous politicians who resigned during that time because they had lied included John Profumo, the British minister, in 1963, and Richard Nixon, the US president, in 1974.

This is quite rare now. Today, politicians who resign when they are caught lying are not the rule, but the exception. Many politicians lie whenever they speak, they know that everyone knows that they are lying, but don't care about it. This is an example of the discredit into which the concept of truth has fallen, which was one of the most important criteria in history, not just for politics, but for science and all forms of human thought.

Singularities

Hal 9000, from the film
2001, a space odyssey

A singularity is a mathematical concept applied to a function of a variable that reaches an infinite value for one or several finite values of its independent variable.

For example, the function y=1/x presents a singularity for x=0, since it is often said that 1 divided by zero is equal to infinity.

Information and intelligence

Von Neumann’s architecture, which applies to almost every computer ever built during the history of computing, builds computers from two clearly separate parts: the processing unit, where instructions are executed, and the memory, where data is stored. Consequently, almost all the programs we run on our computers are divided into two different sections: the algorithm (the executable instructions) and the data that provides the information needed by the algorithm (its input).

Such a clear separation reminds the difference between the two concepts in the title of this post:

A model for ChatGPT

How does ChatGPT work? Suppose we ignore for the moment that ChatGPT uses an artificial neural network, and represent its algorithm in the traditional way. This algorithm can be divided into two parts:

1. Training: ChatGPT is provided with data (text files), which are used to build two data sets:
1. A list of all the words that appear in any of the texts, without repetition, regardless of their order or the number of times each one appears.
2. An array of indices to the word list, reporting the number of times a given word occurs after a series of words. For example, if the following series appears in the texts: time travel, the indices of the words travel, and time will appear in the array, followed by the index of the next word, followed by the number of times that this sequence of three words appears in the set of texts used for training.

Cyberethics

Norbert Wiener

In 1948, Norbert Wiener created the term Cybernetics, applicable to a new technology, which he defined as follows:

The study of control and communication in the animal and the machine

Cybernetics has a lot to do with Robotics and with the use of computers and microprocessors to control and communicate; in other words, to do almost everything we use them for.

But what is being talked about right now, rather than Cybernetics, is Cyberethics: ethical issues related to the use of computers, social networks, and most tools that modern technology puts within our reach.

An Evolving Universe: PopulScience in Book Form

CEU Ediciones has published in book form, under the title Un universo en evolución (An evolving universe), a compilation of about a hundred posts published in this blog and its Spanish version, Divulciencia, over nine years, thematically arranged. Currently this book is only available in Spanish. These are the titles of the fifteen chapters:

1. Introduction (3 posts)
2. The beginning (5 posts)
3. The standard cosmological model (9 posts)
4. The fine-tuning problem and the theories of the multiverse (7 posts)
5. The problem of time (7 posts)
6. Life on Earth and on other worlds (3 posts)
7. The evolution of life (7 posts)
8. Man (5 posts)
9. Natural and artificial intelligence (7 posts)
10. Synthetic life and artificial life (3 posts)
11. The future of man (8 posts)
12. The end of man and of the universe (4 posts)
13. Science, faith, and atheism (14 posts)
14. About science in general (17 posts)
15. Conclusion (1 article)

Julio A. Gonzalo

The book starts by a presentation by Javier Pérez Castells, and a prologue by Julio A. Gonzalo. This is a paragraph from the presentation, which has been highlighted on the back cover of the book:

Manuel Alfonseca presents us in this book with a selection of posts from his blogs arranged by topic. The guiding thread is the story of creation. Beginning with the start of the universe (where he expands more, given his experience and knowledge on this subject), going through the origin of life and evolution, ending with anthropological conceptions of the human being. After that, he turns towards the supposed conflict between science and faith, grouping together a set of posts where he answers to this false tirade. Along the way, he stops to deal with the issue of artificial intelligence, the author’s favorite subject, and devotes a good number of chapters to an optimistical look at the future, albeit not forgetting that everything, together with the story of creation, will have an end. The book concludes with a few pertinent reflections on the limits of science, today considered by many as the only source of knowledge, in the exacerbation of what we call scientism.

Most of the posts (about one hundred) that have become part of the book have been previously published on the blogs Divulciencia and PopulScience, although a few are completely new, others have been built from several consecutive original posts or from those that dealt with the same topics, and the rest, those who have passed directly from the blogs to the book, have been updated.

The same post in Spanish

Thematic Thread on Anniversaries and Organization: Previous Next

Manuel Alfonseca

Another conversation with ChatGPT

As I said in the previous post, last January I did two sessions with ChatGPT from OpenAI. In the second, my questions were directed to the following scientific topics:

• The special theory of relativity, the limit of the speed of light in a vacuum, and the possibility of traveling in time.

Throughout the session I was asking the following questions:

1. Can a particle travel at a speed greater than light?

Answer: According to the theory of relativity, as first described by Albert Einstein, nothing with mass can travel at the speed of light...

A conversation with ChatGPT

Last January I decided to test the latest hit in Artificial Intelligence, ChatGPT from OpenAI. To do this, I carried out two independent sessions (I don’t know if the program connected them to each other). My questions had to do with the following scientific topics:

• The first session dealt with a series of questions about the general theory of relativity, cosmological theories, and the standard cosmological model.
• The second session dealt with the special theory of relativity, the limit of the speed of light in a vacuum, and the possibility of time travel.

After the first session, my conclusions are the following:

Downloading consciousness

If we could download our consciousness in a computer or a robot, we could go on living indefinitely inside the hardware and thus achieve immortality. This is what some futurists and supporters of transhumanism hold. But is there any chance that this will come true?

If this were possible, we could also keep backup copies. Thus, in the event of an accident, the dead person could be retrieved and put back into operation with minimal memory loss. As usual, science fiction arrived there first, as in Cory Doctorow's novel Down and out in the Magic Kingdom.

Science and hate for religion: a personal anecdote

John McCarthy

Since 1995, Internet has become an almost indispensable tool for many people. Our society is increasingly dependent on the global data network, which means that we are increasingly vulnerable. A solar flare, a large meteorite, an overload of energy networks, a major war, can endanger our social structure, making it impossible to use Internet.

In the late 1980s, Internet was just a fledgling network linking together private companies, whatever their hardware and software. In fact, it was one among many networks competing for the new ecological-social niche. One of the most important of these networks was BITNET, which linked all the universities in North America using IBM computers. Later, a parallel network was created in Europe, called EARN, which was soon connected to BITNET.

Will we be able to build conscious beings?

I use the term conscious beings, because the term conscious machines has materialistic connotations that I do not share, while artificial intelligence has been burned and abused by generalist media.

Before trying to answer this question, I want to make a few previous clarifications:

1. We are now capable of building conscious beings: our children. Four billion years of evolution have led to the appearance of a type of beings (us) endowed with self-awareness and capable of reproducing. This was a natural process, which we have not designed ourselves, but has been given to us since before we existed. However, when someone poses the question in the title of this post, it is usually interpreted thus: will we be able to build artificial conscious beings, by means other than natural, devised and developed exclusively by ourselves?

Artificial intelligence or computer algorithms?

John MacCarthy

The term artificial intelligence appears frequently in the media. It is usually used to refer to a computer application that behaves in a way that appears to be intelligent. But is it really intelligent? Or is this a case of meaning displacement, the application of a more appealing term to something not really new?

The term artificial intelligence was invented by John MacCarthy in 1956, in a seminar that took place at Dartmouth College in Hanover, U.S.A. At that time, the participants made exaggerated predictions about the imminent advances they expected in this field, some of which have not been fully performed 65 years later. Now the predictions are even more ambitious, but it is quite likely that none will come true in the short term, and that some will never come true.

Conscience, Self-conscience and Artificial Intelligence

Ramón López de Mántaras

In an article published on 3/22/2021, these words are attributed to Josu Bilbao, head of the ICT area of ​​IKERLAN:

In three years, artificial intelligence will make an intelligent dialogue with machines possible.

We have seen many similar predictions since the term Artificial Intelligence (AI) was invented over 60 years ago. In most cases (if not all) those predictions have been too optimistic. Is the same going to happen here?

I have consulted with one of the world's leading experts in the field of the automatic analysis of natural language, a technique used to implement the type of applications referred to in the article I am commenting, and he told me this:

It depends on how "intelligent dialogue" is defined. If it is restricted to a specific domain, for example, controlling your home automation systems using Google Home, that can be done now with a success rate above 70%, while ten years ago it was science fiction. If you want to have a philosophical conversation where your interlocutor understands what he is saying, rather than generating texts from a language model, this is too optimistic.

The golden rules of science fiction and my own work

Isaac Asimov

These are, according to Isaac Asimov, the two golden rules of good science fiction:

1.      Any scientific claim (even a prediction) must always be compatible with current science. It would not, therefore, be good science fiction a novel where the squaring of the circle with a ruler and compass was achieved, because it has been shown mathematically that this is impossible. Or traveling through space at a speed greater than that of light, without using some scientific trick that makes it possible to achieve it, such as going through the world of tachyons.

2.      Predicting social consequences is better than predicting technical advances. Thus, in a hypothetical novel written in the nineteenth century, predicting the parking problem would be much better than just predicting the car. An example of this kind of good science fiction, mentioned by Asimov in this regard, is Robert Heinlein's short story Solution Unsatisfactory, written in 1941, which not only predicted the atomic bomb as a means of ending World War II (in which the United States were not yet taking part), but also predicted the subsequent equilibrium between the great powers and the permanent threat of a war of extermination.

To be exact, these two laws are my elaborations of what Asimov wrote in two popular articles published in The Magazine of Fantasy and Science Fiction: Future? Tense! (1965) and O Keen Eyed Peerer into the Future (1974). To be even more exact, Asimov's laws were three and, similarly to his three laws of Robotics, he called them the three laws of Futurics.

Science and Science Fiction: mutual influences in the 20th century

H.G. Wells

NOTICE for those of you who have subscribed to the posts of this blog by email: Google has announced the end of "feedburner", its notification service, which was being used by this blog. From now on, another service will be used: "follow.it". You may initially receive two notices, one from each service.

Halfway between the 19th and 20th centuries, the British H.G. Wells also practiced the literature of scientific prediction, although his predictions are usually longer-term than Verne's, more remote from the technique of his time, so very few have come to be carried out. We still don't have The Time Machine and can't see it anywhere near. We cannot make artificial men by vivisecting animals (The Island of Doctor Moreau). Nor can we make ourselves invisible (The invisible man). And fortunately the Martians have not invaded us (The War of the Worlds). In 1938, a radio dramatization of this novel by Orson Welles caused a mass panic in the United States. Despite the boom in science fiction at the time, the public was still as credulous as when it had been fooled a century earlier by the articles in the Sun.

Wells also wrote (of course) about a trip to the moon and populated our satellite with intelligent giant ants, although the procedure to make the trip is more imaginative than, and as impractical as Verne's. But his great scientific foresight success is the novel The World set free (1913), which not only anticipated the atomic bomb, but also influenced its practical realization, stimulating Leo Szilard's research on the neutron chain reaction. Although Wells also made a tremendous mistake in this novel, by predicting in 1913 that the First World War would start in 1956. But as he said a few years later: [I have] always been... a rather slow prophet. You can also see my review of this book in Goodreads.

Artificial neural networks

Perceptron

One of the oldest applications of artificial intelligence is based on the simulation of animal nerve cells (neurons). Neural networks are made up of many interconnected components, and they are capable of some computational activity, although the neurons that make up these networks are usually quite simplified, compared to those that are part of the nervous system of human beings and other animals.

This type of application has been the subject of multiple exaggerations and unusual predictions. These networks have been said to be capable of solving the most difficult problems, NP-complete problems, like the traveling salesman problem, and similar ones. A normal program can solve these problems, but the time needed grows exponentially as a function of the size of the problem, while a neural network can solve them in a short time. To some extent this is true, provided that we bear in mind that the solution obtained is not necessarily the best, but only an approximation, which is often sufficient for our needs.

Algorithmic censorship and diversity in scientific research

Manuel Cebrian

Manuel Cebrian started working in USA at MIT, and after a long journey that took him to the West Coast of the United States and then Australia, he returned to MIT and is now in Berlin. He became famous thanks to having won two important competitions organized by the government of the United States, related to the use of social networks to solve more or less complex problems:

• DARPA Network Challenge (2009), which offered $40,000 reward to the first team that managed to discover, in less than 8 hours, the location of fifteen red balloons distributed in different locations in the United States by Pentagon personnel, using a social network of their own creation, built for one month before the competition date. Although they received many news about false sightings (fake news) Cebrian’s team managed to win the competition, against over 9000 participating teams.
• DoS Tag Challenge (2012), which offered $5,000 reward to the team that managed to locate five actors, identified by their photograph, who acted as though they were five criminal suspects that would remain visible for 12 hours in five European and American cities: New York, Washington, London, Bratislava and Stockholm. Although they just managed to locate three of the five suspects, Cebrian's team won the competition again, despite the unethical behavior of other participating teams, one of which copied their website to deceive possible informants, so they’d send their information to the web of a different group.

During his stay in Australia, Cebrian worked on forecasting the negative effects of natural catastrophes by analyzing data provided by social networks (essentially Facebook and Twitter). The bad news is that, increasingly in recent years, algorithms that estimate general interest of news and messages in social networks are influencing whether and how those news and messages are propagated over the networks. These algorithms decide whether the messages and news will be more or less seen, and whether the users will stay more or less time on the platform.

In light of this, Cebrian and his colleagues transferred their research on social networks to the way in which Artificial Intelligence (AI) affects human communication and cooperation. The theoretical analysis of computational models inspired by the changes made in the large platforms made it possible to conjecture that the functioning of social networks can be affected by algorithms capable of enhancing or weakening information. As a result of their use, messages and news originated by certain users reach a smaller number of followers, which could be considered a form of algorithmic censorship.

Concerned about the growing trend towards control of the Internet by large companies, Cebrian and his team have made a thorough analysis of the publications in the field of Artificial Intelligence, and of the cross-references between this and other fields of science and the humanities, between 1950 and 2017. It has been found that, although the number of publications on AI has increased progressively, the mutual impact with other sciences seems to be decreasing. Just four fields (Computer Science, Mathematics, Geography and Engineering) maintain a trend higher than a randomized cross-data distribution, and just one (Computing) is increasing, although at a level below that reached during the seventies and eighties.

Another result obtained by the analysis described in the just mentioned article is that research on AI is increasingly dominated by a few research institutions, and the most cited publications appear in a small number of journals and conferences. This decrease in diversity, which has reached 30% since 1980, affects authors, articles and citations, and suggests that cross-citing research hubs may exist, with well-defined preferences. On the other hand, the preponderance of large companies in these hubs (Google, Microsoft, Facebook) could lead to the goal of research changing, from finding solutions to important technological problems for human beings, to a new situation where the objective is collecting customer data and selling them to advertisers, thus controlling the purchase impulses of society.

Let us look at one of the most important and worrisome conclusions of the article:

The gap between social science and AI research means that researchers and policymakers may be ignorant of the social, ethical and societal implications of new AI systems.

Martín López Corredoira

This decrease in diversity, combined with censorship of what does not fit the orthodox model, also affects other fields of science. Consider, for instance, the field of cosmological physics (cosmology). The publication with the highest impact is now arXiv, to the point that it is currently very difficult to publish there, unless you belong to an important institution, and totally impossible if the article differs in any way from the ΛCDM model, the current standard for cosmology, as denounced in this book:

Nobody should have a monopoly of the truth in this universe. The censorship and suppression of challenging ideas against the tide of mainstream research, the blacklisting of scientists, for instance, is neither the best way to do and filter science, nor to promote progress in the human knowledge. The removal of good and novel ideas from the scientific stage is very detrimental to the pursuit of the truth. There are instances in which a mere unqualified belief can occasionally be converted into a generally accepted scientific theory through the screening action of refereed literature and meetings planned by the scientific organizing committees and through the distribution of funds controlled by "club opinions". It leads to unitary paradigms and unitary thinking not necessarily associated to the unique truth. 

The same post in Spanish
Thematic thread on Natural and Artificial Intelligence: Preceding Next
Thematic Thread on Science in General: Previous Next

Manuel Alfonseca

Fred Saberhagen versus the Turing Test

Alan Turing

In 1950, the English mathematician and chemist Alan Turing tried to define the conditions so that it would be possible to affirm that a machine is capable of thinking like us. For Turing, this will be achieved when the machine is capable of deceiving human beings, making them think that it is one of them. This test is called the imitation game. I have talked about this in a previous post in this blog.

In 1956, Arthur Samuel of IBM built a program to play the game called draughts or checkers. The program kept information about the moves in the games it had played, which was used to modify its future moves (in other words, it learned). In a few years, after playing many games, the program was able to defeat its creator and played reasonably well in official championships.

That same year, during a summer course held in Dartmouth College, John McCarthy and other computer pioneers coined the term artificial intelligence. Getting their hopes too high, they predicted spectacular advances for the next ten years, which did not take place in the time envisaged, but much later. I have also written about this in another post.

In 1963, science fiction writer Fred Saberhagen published the first story of his famous series about the berserkers, autonomous and intelligent space fortresses created by an ancient extraterrestrial civilization to exterminate intelligent life wherever it appears in the galaxy. This story, titled Without a thought, is an answer to the Turing Test and a brake to the unbridled hopes of the inventors of the term artificial intelligence. This is the plot of the story:

Freedom and prior censorship

Wikipedia Logo

On July 5, 2018 the European Parliament rejected, by 318 votes against 278, the proposal for the Copyright Directive in the European Single Market. In the days leading up to this vote, there were many public and private activities in favor and against the proposal, which after this defeat will have to be debated again in committee, probably with amendments. The most controversial points of the proposal, those that gathered most rejection, were incorporated in two articles of the regulation:

• Article 11: Establishes what has been popularly called the Google tax. It makes it compulsory, for those responsible for web pages, to request permission, and if the copyright owners wish, to pay a fee, for including a link to a news or copyright owner that has appeared in any of the media. The most favored by this article are not individual authors, but mass media (especially the press on the Internet), the main defenders of this measure.

The MEPs who defended this article argue that it does not affect individuals or the Wikipedia, although the latter felt so threatened, that it declared a strike for the first time in its history, so that access to the Spanish, Italian and French versions of the Wikipedia was closed during the day before the vote. The problem is, this article may be expressed so ambiguously that, although just now may not apply to individuals or to Wikipedia, there are no guarantees that in the future this cannot be done.