I'm really pleased to introduce today, multi-piece. He's going to talk about
connecting GPT with your data and how to avoid
hallucination, which is something that we obviously talked about in the Q&A about an
hour ago, Monty is the co-founder and CTO at deepset,
where he builds Haystack, and deepset Cloud to enable
developers all over the world to use NLP, effectively in their business
applications previously, he conducted NLP research at
Carnegie Mellon University.
And was a data scientist for multiple startups. He's been crafting, NLP, applications
for all kinds of businesses for more than eight years now and is convinced that the
development workflow is your orientation are key criteria for
successful in LP projects. So multi please take it away.
Thanks, John. Thanks for the kind intro. Yeah, I'm super happy to be
here today and share a few of our thoughts on llms.
Welcome to the session and I think we are
all aware that the lamps are the future, right? And that they actually
unlocked already, many exciting use cases. We already
heard about a few in the sessions earlier today.
Still, I personally feel that there's sometimes two
realities out there. There's this
Crazy bus world. When I my Twitter feed, for example,
where everything seems possible and so easy. And then there's
Enterprise reality when I talked to Engineers product
managers out there in the industry. So, if we look at the
title of this whole event today, my talk will be more on the
pitfalls side of things. And I want to share some of our learnings
some of the challenges we basically saw with our Enterprise customers
when building a la application.
Ins for production use cases.
To give this whole session a useful and positive twist. I will of course, not only share
pitfalls but also some pragmatic tips to actually overcome
them.
So the next 20 minutes, I will give you an overview over the most common
challenges when building applications then highlight
one key method that helps a lot to overcome some of these
challenges and then walk you through some additional smaller
tips on what you could actually do to optimize things even further.
So, let's get started with the most common challenges. We observed with our customers
when building element education's
Number one, most of these high-value use cases, we see in the
Enterprise includes some kind of internal company data
so that is data that is not available in the public Internet.
It's often from special domain could be for example,
aircraft maintenance documents could be legal contracts
or financial reports and the typical question
that comes up then is how can we tune the model now to
our
Data. How can we teach it about our internal knowledge that it doesn't know yet about?
Change number two is actually closely related to the first one when our
use case. Now relies on this internal data. How can we make sure we stay
in control of it? We must be safe. Must be secure.
Maybe our company policy doesn't allow sharing this kind of data with third parties.
Number three. If you overcome these first two challenges,
the next big question, question is typically around quality insurance. So once
you want to go and production, think we all saw these models hallucinating
simply making up answers. And if your application is being exposed
to customers. For example, this could be extremely damaging to your
brand. So, how do you know if there's a safe
enough to deploy? How can you assess this quality?
I could probably talk about each of those challenges for hours
today. I just want to focus on Challenge 1 and 3 and get there, the
key ideas across. So let's start with the basics.
What is actually an hallucination? Why does it happen?
Well, the Lambs are often wrong with the answers and even worse, they are
confident about it. So some of you might remember this bank that
used to exist Silicon Valley Bank and if you
ask me LM, if Silicon Valley Bank collapsed you can easily get an
answer like this. No, it did not collapse its successful. Financial Service
Company dada.
This is what we call hallucination. The model is simply making up fake
reality.
So let's try something else. What if we asked, why did Silicon Valley Bank collapse?
All of a sudden the model switches its opinion. Now as we actually collapsed
great
It's even giving us some arguments why the bank failed but wait,
it's mentioning something around subprime mortgages and financial
crisis of 2008.
I want to see very bad with remembering dates and timelines, but even I'm
pretty sure that SV B was happening this year
and not back then in 2008
and it was actually related to Rising interest rates and a bank run
not subprime mortgages.
So this kind of behavior is what causes problems When developing an llm
application. The model is not stable. In its
opinion, it's giving wrong answers and it's often very
eloquent in. Its argumentation comes, can sound very convincing.
This makes it really hard to spot. These kind of hallucinations as a user.
Where do these hallucinations now come from. Well, in this case, it's probably rather
easy. As we be happened this year, this LM that I asked here, only
saw training data on 2 September, 20 21. So
it's internal knowledge, is simply outdated. It could most likely be
fixed by training another model on more recent web data.
Okay, cool. So I hallucinations
all your problem when we asked about this kind of recent events,
Unfortunately not.
So let's ask Deputy about something in 2008. Let's ask about the number
of model 3, weekers that Tesla produced in the first quarter of
that year.
The answer that we get here, sounds convincing. It's even mentioning a
source. The shareholder letter. However, if you open the
shareholder letter, you find that this number is unfortunately completely wrong.
Instead 9700 vehicles of that type were
produced
So why did this model hallucinate here?
Either they did not see any information about this at all at training time is
quite unlikely. In this case as it mentions explicitly this kind of shareholder
letter or well the more likely option this case.
The model actually has seen some related information but mixed up, it's not confident
enough. Maybe hasn't seen enough examples of it. It's
it's guessing.
And indeed, if you look through the shareholder letter, this number of 34,000
cars that the model brought things up here is the total number of
vehicles produced in the quarter, not for model free.
So now that we know what hallucinations are, how can we actually reduce them
to ensure quality and production?
And let's not forget about our Second Challenge. How can we teach the LM new
information for use case is about this internal company data.
Turns out there's one key method that can actually really help with
both of these challenges, it's called retrieval-augmented.
Let Me Explain you. The key idea of this method,
Let's first see how an answer gets generated by an llm in the standard way?
So first, you have your question.
This is typically converted to a prompt which contains still your
original question. But then also maybe some optional text around it
to instruct the model. What to do actually maybe in what style should
respond
This prompts then sent to an llm and this generates the
answer that we then see simple.
Now let's look what changes with this approach of
retrieval-augmented, a ship. We still have our question here on the left.
We still have a prompt that gets fed to our LM.
However what we do now is inserting some more information or
prompt what you see in green here. So what we want to
provide the llm with some more useful information that helps the
model to answer our question in a really truthful way. So in our
example, of Silicon Valley Bank, this could be news
articles or analyst, reports that explain this
collapse of the bank.
Of course, we need some automatic automatic way of searching and
inserting these relevant documents in here. Can't do this
manually.
So what we do is now connecting our pipeline to some
external database and use a so-called retriever
model to find us a few relevant documents that we can then
insert basically into or prompt. So think of it as a search
first search relevant piece of information and then insert it into our prompt.
And when we do that, we can actually solve now several problems at once.
First of all, we reduce hallucinations as the model will now
ground its answers on some actual information. The documents.
You can also teach now, the LM new information so that it stays up to
date and also becomes aware of our private company data that
we might have in this database.
Last but not least, it helps with explain ability and verifiability,
for users as a user. I can now very easily browse the documents
behind my generated, answer. Some to what we were earlier in the talk
about being, I can verify that these
answers make sense and where they come from.
So, let's now have a quick live
demo and see this in action.
So this is a small demo that you put together. It's on hacking face basis,
so all of you can actually access it. It's using Haystack
open source framework under the hood or coat is also available here on the
files and what you can do here now, is basically asking questions around
the Silicon, Valley Bank collapse. So we
have here already, one example, did SVP collapse and then we see basically
two different answers, one using plain to PT
and one using
You with retrieval-augmented, and can switch. Basically.
The data set that is used, you can either use a static news
data set some articles, or you can do live web search to augment your
upfront. So let's maybe try one of these prairies,
run it.
And yeah, laughs here. We see again some some plain answer with
hallucination and down here. We should.
Hopefully see also yet. We should see actual
the SED collapse due to a bank, run caused by VCC and
Founders withdrawing their funds to the data
headwinds from continued higher interest
rates. So that seems like more more truthful
and we can even browse The Source, the
Text behind it, that was used to generate results.
So, have a look at it play around with it. It's out there.
You cannot only play around with it but there's also linked in the slides,
the code for the demo and tutorial. That actually explains
you walks you through how we can build this easily with open source code yourself.
Something like this should probably make sure some pipeline take you
an hour to build yourself.
So now that you've learned about one key methods, retrieval-augmented ssion, what can we do on top
to have less hallucination? We can actually do a lot.
Let me use the remaining few minutes now to share some quick tips and
general directions that you can then further explore yourself or we
can discuss them in our Q&A session, the breakout room.
So my first tip would be invest some time in optimizing your prompts.
A few things that we found particularly helpful here in practice,
allowing the model to say. I don't know, often rather return
nothing than a wrong answer. Depends on your, not on your use case. But in many
use cases, you rather prefer this
And you can actually instruct simply instruct your model to do. So one example
prompt you see her here on the slide in green, that's basically the
instruction that the model should should say. I don't know
when the information is not grounded or the answers of ground actually in the documents.
Two things that are less obvious, but work establishing astonishingly. Well, in practice,
you can instruct the model with the different tasks that you actually have in mind. So, for
example, formulate, it rather like a summarization task and less
like a question answering task in our example here,
You can also formulate the document context
like the opinion of a person so these are lies nice
little tricks to ground the model even more in this
context that you provide in more, in the documents that we give it to give it to them.
There's a nice recent paper. I linked also hear from zoo at all, where they show that
this opinion based approach actually reduces hallucination quite a bit
and for the summarization task trick, this is
based on our own experience in a recent customer.
A project that we actually made the observation that you have generally less
hallucination in summarization tasks. And we kind of hijacked the snow
for Q&A tasks as well. My intuition here would
be that probably related to how this models were trained
reinforcement learning on this kind of human feedback and
maybe the labelers were kind of strict off for summarization task the
label slightly different that the penalizes
hallucinations anymore.
So there's much more you can do of course, but that was for now
prompt engineering at a glimpse that's now spend
a few more minutes on other useful directions. What else can we do?
Beyond prompt engineering?
Well, so far we optimize the input to the LM. If you look at this chart
here, we basically just what goes into the LM is there,
maybe or something you can do after the LM when we got back to the generated
answer.
Yes, for sure. There's something and one of these helpful things that you can do there
is so-called self-reflection. This basically idea
of letting the model reflect on its own generated
response, and give the model a chance to correct it. Let's look at a
basic example.
Let's ask the LM who the CEO of Twitter is. And as we know,
this has rather recently changed. You also Supply directly the
information that is actually needed to answer our question. We let the model already
know that. Even mask is the actual owner and CEO.
Just as you would do with a retrieval-augmented, a Sheen approach.
If we look now at the response, this is a bit disappointing. The model are
ignored the extra information, we provided it wants us about its
internal knowledge cut off, which is nice, okay, but it actually did not
really answer our question so what can we do to now
improve this answer?
We can actually ask a follow-up question. In the simple example, we can just ask if
the, the answer is really reflecting the information we shared with the
model and as you can see it helps. Next response is basically
containing what we're after the model corrects itself, kind of
course not. Do these follow-up questions, mainly as a user, but we can also
automate this in our application. If you automate it, this then, typically what you call
self-reflection
Would add a step in your pipeline. That asks these kind of questions
automatically?
Okay, so now we talk a lot about practical ways of
reducing Allison Nations. But how do we know that we actually
improve? How can we be sure that the level of hallucinations is
acceptable for all production application?
Well, ideally you would have a way of detecting and measuring hallucination.
Just like any other metric. We have for machine learning
models.
Unfortunately it turns out that this is not easy. It's I would say, one of the
probably biggest unsolved problems in the generative AI space and the
Super Active area of research.
It's not like that. There are no methods out there are actually quite a few.
Just honestly I haven't seen them working very well yet.
They are developing.
You can categorize these approach that are out there in three buckets
and in each of these packets, if we see great advances and
things are developing. So, let me walk you through quickly, through these
markets and different approaches. There are statistical metrics. Think of
it as classical machine learning metrics, like F1
Rouge, material, or very easy to compute
relying on some kind of statistical formula.
Benchmarked a few of them recently Unfortunately they are not working very well
for detecting hallucinations. The main problem. They don't
correlate well with human judgments so human
would flag something differently than this metric. They would take
different answers as hallucinations that this metric doesn't get
Which brings me to my next Point collecting. Human feedback is currently probably the
most precise thing you can do. You can ask some users to
check the predictions of your prototype Pipeline and kind of take
all hallucinations. This, of course works. You get some
Metric out of it with right tooling. This doesn't take too
much time either and we can actually do this to actually do this quite a lot
with our customers.
However, let's do quite some mental
effort is involved there, right? And maybe you can do it once or twice before
going to production, but what then, how can you continuously
monitor or hallucinations of your life application? So
somehow we still really want an automated way of detecting hallucinations.
And this is where the third project, third approach out
there fits in and it is quite promising which is model
based detection of hallucination.
This approach, lets you detect hallucinations automatically by just
using another Machinery model that is specialized for this task.
The only problem so far, there's no really got good model
specialized for it out there, and that's why we decided to train such a model
ourselves. So let me give you a very quick sneak preview of
what we work on right now there.
So the idea straightforward the model that we train takes, two inputs
generated answer and then some
ground truth data that can compare to. And in the case of
retrieval-augmented a Sheen, we have this actually for free. We have our
retrieve documents which already contain the kind of Base information
and we can use this as as kind of ground truth data.
So the mother gets those two text inputs and then
classifies if the generated answer from the LM is actually grounded in
the retrieve documents or not, and it returns a score
between 0 and 1. We called faithfulness score,
And with that score you can now automatically evaluate every answer, our model generates
and depending on our use case route questions
differently or others differently at a self-reflection loop or
whatever you want to do with this answer.
So there's more still in the making on our side, but be assured. Once we have a
model here, we will open source this so stay tuned.
And with that, I'm also already coming to an end of this short session.
I hope you saw threat of Hallucination is
real and it's not only about outdated information.
We also saw how retrieval-augmented
agen can actually help with reducing this hallucinations but also
tailoring, your responses towards their own data and I
shared a few directions, like tips. What you can do further
Last but not least, I also want to stress that this is not only holds
four simple pipelines with a singer LM call. But
also for more complex systems like agents with often dozens of
Adam calls to make these agents actually robust. It's even more important to
reduce hallucinations and Tack them on each other.
Yeah with that. Thank you.