Last January I decided to test the latest hit in Artificial Intelligence, ChatGPT from OpenAI. To do this, I carried out two independent sessions (I don’t know if the program connected them to each other). My questions had to do with the following scientific topics:
- The
first session dealt with a series of questions about the general theory of relativity,
cosmological theories, and the standard cosmological model.
- The
second session dealt with the special theory
of relativity, the limit of the speed of light in a vacuum,
and the possibility of time travel.
After the first session, my conclusions are the following:
- ChatGPT
returns the information little by little. One must extract it
by means of carfully chosen questions. In fact, the initial answers, in
addition to being incomplete, can be wrong, and it is not uncommon for
several successive answers to contradict each other. Let’s look at an
example:
First,
I asked about Friedman's equations. The answer gave me a version of the
equations without the Λ
term. I then asked about the cosmological constant, the program
apologized, and offered me the full version of the equations, adding as an explanation
that the effect of the Λ term is to cause
an accelerated expansion. When I pointed out that Λ
can be positive or negative, I got a new apology, with the explanation that the
observable data seems to indicate that Λ is
positive.
- Sometimes
ChatGTP makes elementary syntax errors. To a question about
Fred Hoyle’s cosmology, I was answered with three paragraphs, the first of
which contained these words:
While he was a proponent of a steady-state model of the
universe, which proposed that new matter is continuously created to form new
stars and galaxies, and the universe has no beginning and no end.
This
sentence is subordinate and therefore has a beginning but has no end. In my
next contribution I simply pointed out that your
first paragraph has incorrect syntax. The program apologized,
tried again, and offered the following version:
To clarify, Fred Hoyle was a proponent of a steady-state model of
the universe, which proposed that new matter is continuously created to form
new stars and galaxies and that the universe has no beginning and no end.
Which is not subordinate and therefore is correct.
- It
is not difficult to force it to back down. For example,
to my questions about the standard cosmological model, the answer was
this:
The standard cosmological model, also known as the Lambda-CDM
model, has a strong record of making accurate predictions about the universe.
After
which it gives, as the model’s predictions, measurements
of the cosmic microwave radiation, the large-scale distribution of galaxies,
the observed abundance of light elements (meaning hydrogen and
helium), and accelerated expansion.
Predictions about dark matter are also mentioned. I then pointed out that most of
these are not predictions, but adjustments
to previously observed data. It apologized, accepted the correction, and pointed
out that the model has made predictions not yet confirmed, such as those
related to dark matter and neutrinos.
Along this session, I detected the
following:
- The
answers were somewhat repetitive. During the
conversation, the program answered sometimes with practically identical
paragraphs.
- The
program adapted to me: it always agreed with me,
never contradicted what I said, and increased the scientific level and correctness
of its answers as I pointed out deficiencies.
 |
| Fred Hoyle |
My conclusion was that this program only makes sense when the questioner is an
expert in what is being asked. Otherwise, it is very likely that
the questioner will get an incomplete idea, or even a wrong idea, about the
scientific questions consulted.
At the end of this test, I thought it
would be interesting to find out what would happen if ChatGPT were asked misleading
questions, which would lead it down a path where it might get lost, and perhaps
run into contradictions. That’s why I did the second session. But we will talk
about this in my next post.
Thematic Thread about Natural and Artificial Intelligence: Previous Next
Manuel Alfonseca
No comments:
Post a Comment