- Okay we're now about to talk about data races.
And what a data race is is when you've got
at least two paths of execution, like two Go routines,
accessing the same memory location at the same time,
where one is doing a read,
and the other is at least doing a write.
I mean they both could me doing writes too,
and that would be a data race.
And that is bad, okay?
You cannot be mutating memory okay?
It's two paths of execution at the same time,
we're gonna have data corruption.
And this is where synchronization comes in.
There's two things here, synchronization and orchestration.
We're focusing on synchronization
right now in this data race section.
The best way to think about synchronization
is if you went to Starbucks okay,
and you got in line 'cause you wanna get some coffee.
And now you're in line,
you're waiting for your turn to get up to the counter.
Anytime Go routines have to get in line,
that is a synchronization issue.
But once you get to the counter
and you start talking to the person at the register,
you now have an orchestration issue.
We're having a conversation, we're exchanging money,
there's data going back and forth.
That is orchestration.
And we have these two problems,
synchronization and orchestration.
And your biggest job is to know
when it's a synchronization issue,
when do we have to get in line?
And when it's an orchestration issue,
when is there a work flow going on.
Now again, a data race is when
you have two or more Go routines
where one's doing a read and one is doing a write to the
same memory location, that is really bad.
We cannot have that.
We have some other really special things
also happening at the hardware level.
We really have value semantics at the hardware level.
We've gotta appreciate this when we
start writing multi-threaded software
because it can really hurt our performance.
Our cacheing systems,
though they're helping us try to reduce latency
to main memory access,
you can actually be thrashing memory
as well if we're not careful.
Now I am praying that you went back
and watched this entire video,
that you didn't start here in concurrency.
Because in the array section, in data registration,
we really talk about cache lines and how these things work.
And it's gonna be very important
for you to watch that stuff before you come here.
I'm gonna assume that you've seen
all the array and cache line stuff as we get going.
Now, let's talk about the cache coherency problem
and how these value semantics could come in and hurt us.
And then we're gonna talk about something
even more interesting to me, which is false sharing,
all around this idea again of the cache coherency problem.
So let's start with our four core processor.
Now let's say we have our four core processor, here it is.
And we've got our core one, two, three, and four.
And we know that every one of these cores
has it's own L1 and L2 cache with the shared L3.
Now I said that the hardware has
these value semantics because what I told you,
value semantics means
that we're always operating on our own copy of the data.
So imagine this.
I've got a loke,
I got a global variable in a sense, we call it counter.
Starts at zero.
And I decide to launch four Go routines, okay.
Let's call this Go routine zero, Go routine one,
Go routine two, and Go routine three.
And these four Go routines are gonna run in parallel.
We're gonna get them to run on their own P,
on their own M, and therefore they're gonna
execute against their own individual core.
Now, if we want every Go routine to
be remodifying and writing C,
we now have a synchronization issue.
We've gotta make sure that only
one Go routine reads, modifies, write at the same time.
This is where we're gonna need things like
atomic instructions,
which are at the hardware level, and mutexes,
which are just above that, okay.
I'm gonna show you soon how to
leverage your atomic and your mutexes
to create these atomic instructions,
or make multiple statements in time.
But let's get back down to the hardware for a second.
I want you to remember something here.
If every Go routine is going to
take an opportunity to read, modify, write C,
then a copy of C has to be brought
in to each core, there it is.
That's our copy of C, right?
The cache line that C is on wherever
that is, is gonna brought in.
Now this is where things get interesting.
If we, let's say we're gonna use our
entomic instructions since it's
counter here, to do synchronization.
That's fine.
That means that when Go routine zero
decides to perform a read, modify, write
to turn this from zero to one,
then through the magic of hardware
the hardware's gonna say, okay G1, G2, and G3
if you plan on doing this,
I'm gonna put you on hold okay,
because G0 on Core one
is doing it's thing,
and we're gonna make sure that that happens.
But look at how we're also
gonna be able to do thrashing memory here.
If we go ahead
and increment this cache line to be one to zero,
then once we modify this cache line,
move through the magic of hardware as well
through these snooping protocols,
we're gonna be marking these cache lines dirty.
Because we've now just updated this one,
we're gonna probably,
those results will go back there in main.
But we've updated this,
this is now really the current right state of what C is.
C isn't zero anymore, C is one.
But, our copy of the cache line
still has the representation of zero,
and we've marked it dirty.
So now when Go one, Go routine one gets a chance,
right, through synchronization, to modify C,
it's gonna identify that the cache line that
it has is dirty, it's not the true,
it's gonna have to go out into main memory
to go get a copy of the current one.
And now this isn't gonna be dirty anymore, right?
And then it's going to increment this
from one to two, which is brilliant, right?
And we're gonna have synchronization, too.
But once we do that,
it's gonna mark that cache line dirty,
and it's gonna mark this cache line dirty again.
Whoa.
Now remember these increments
are probably happening across nanoseconds of time.
So as the synchronization is in place,
which is still a cost, right?
We're gonna be constantly thrashing through memory
on every single increment.
That 107 clock cycle latency hit
to bring us back in
because we have shared this value across all four cores.
And I you've got a 36 core processor,
this is gonna be even worse.
Especially if you've got multiple processors
in there with multiple cores.
All of that processor communication
that we were talking about internally here
also has to happen outside.
This is very, very nasty,
but be very careful about global variables
and global counters in a multi-threaded situation,
because we're not really just referencing
this one value and updating it.
Remember we've got value
semantics at the hardware level to reduce access
to main memory costs,
and therefore as we share data, right,
accessing data again across these cores,
we've got some other problems.
But there's also something here called false sharing,
which is super interesting.
And false sharing occurs when you don't really
have a synchronization problem,
but we still have the cache coherency issue.
Imagine we said okay, I don't wanna share,
I don't wanna increment the same
global variable cache on my cores.
Let's bring our processor back into play here.
So here's our processor again, here it is.
Core one, core two, core three, core four.
L1, L2, and our L3.
And this time what we say is even though
we're gonna be running a Go routine across,
Go routine zero, Go routine one,
Go routine two, and Go routine three,
we're not gonna have them increment the same counter.
What we're gonna do is have them
increment their own counter.
Zero, one, two, three.
Let's just say we did it like that.
Now, we don't have a synchronization issue anymore.
The address for index zero is
completely independent of the address of index one.
So, I don't need any any atomic instructions or mutexes.
When G0 wants to increment it's
counter from zero to one, it shouldn't effect what
G1 wants to do, which is from zero to one.
It won't effect necessarily what this
wants to do zero to one, because they're
all accessing their own independent memory location,
and we don't have a data race.
We don't have two Go routines trying
to read and modify, right, the same
memory location at the same time.
However, remember our value semantics at the hardware level.
Just because we don't have a synchronization issue,
again doesn't mean that our cache coherency
problem doesn't exist.
This array is still going to fall
on the same cache line, that 64 byte cache line.
So even though the address was zero, it's
not the same as address one.
Well the cache line for the entire array
is going to be duplicated across all of the cores.
And so though we don't need an atomic instruction
when G1 wants to do a read, modify, write,
against this, or G0 and G1 wants to a
read, modify, write against that.
Even though I don't need the atomic instructions
anymore, right, or the mutex, however one we choose.
However, what's gonna happen is
when G0 read, modifies, write index zero,
it's still going to mark all of these cache lines dirty.
Fact, every one of these increments will
continue to mark these cache lines dirty.
We're gonna have to come back out, then come back in.
So we're still gonna be taking the
thrashing of the memory because
these counters are sitting on the
same cache line (laughter).
Multi-threaded software is complicated.
Especially when you've got data
that you're working with,
that is next to each other
even though you're not necessarily conflicting.
That's the false sharing.
You've got data access patterns
to memory that is next to each other
even though they are unique, but they
fall on the same cache line.
Look, we essentially are sharing the data.
So as we continue to learn about data races,
and synchroniZation, and orchestration.
And as we start moving into the
tooling and do some more live coding
I'm gonna bring this stuff back.
Because any time you have a global variable
you've gotta worry about synchronization,
because if you've got multiple paths of
execution, you can't have a read and
a write happening at the same time.
But we also have to worry about
data access patterns even if the
data is unique but is next to each other,
because we don't wanna deal with either
false sharing, like this is.
But we don't wanna deal with cache coherency
problems where all we're doing is
thrashing through memory because
copies of that data is being leveraged,
re-modified really that's our problem,
across all these cores.
So we're gonna bring back the cache coherency
and the false sharing issues,
as we continue in this class I'm gonna bring them up.
Because again,
when we're writing multi-threaded software,
we've got to be able to see that type
of linear performance growth as
the number of cores increase.
We wanna see this type of growth in our performance.
If you start seeing things like this,
you know whatever that is,
that means we're not being mechanically sympathetic.
It could be our cache coherency issues
where we're thrashing through memory.
So we're gonna keep this stuff in mind as we continue.
And one of the first things we're gonna do
is just we're gonna very simple data race,
and look at ways that we can correct
the code to make sure that we
have no synchronization issues.