- So benchmarking in Go, is incredibly powerful.
I'm going to show you some aspects
of basic benchmarking now,
and then when we get into profiling,
I'm going to show you some more.
But, let's just start with a basic benchmark here.
Let's look at the mechanics
and look at how we can write these
and run a couple of them right there on the command line.
So, what I'm starting out with is on line 17.
In fact, look at line 17 and line 28.
The idea of this benchmark is to do a CPU profile
of how fast
the S print or S print f function run.
And one thing about benchmarking, is that we gotta really
make sure that we're not guessing.
We gotta make sure that the benchmark we're writing
is accurate.
And one of the things I worry about,
is since this code's going to be compiled,
and the compiler is capable of throwing dead code away.
With the SSA backend, it is capable of identifying,
like this scenario, you call a function like S print,
and S print doesn't really mutate state.
It's using value semantics.
It gets its own copy of the string,
returns a new string.
So, if you don't capture the new string coming out
of S print or S print f,
technically the compiler can identify that,
let's not waste any CPU cycles making this function call.
And suddenly, you're trying to do a benchmark like this
to figure out what's faster, S print or S print f.
And it's blazing fast for both,
because you didn't capture the output.
So, what we're doing in this test
is also making sure that the code
is not getting thrown away,
but also that we're using code the same way
as accurately as we can, like we do in production.
Now, here's the test.
I want to know what's faster, S print or S print f,
for the string, hello.
My guess is, is that S print is going to be faster
because we're not doing any formatting.
And remember, any time you guess about something,
you got a good chance you're going to be wrong,
but this one's going to be obvious, right?
I'm just showing you a basic test, S print versus S print f.
Now, look at the structure of the test,
we've got a function named Benchmark,
this time it takes a testing B pointer,
not a testing T, a testing B pointer.
It's the word Benchmark, we have the same rules,
it has to be exported S print.
'Kay, S print also becomes the name for that particular
test Benchmark function.
And the big thing here is the loop.
Now, the way the loop works, is the first time
the testing framer calls this Benchmark function,
b.N is going to be one.
And then it's going to increase b.N over time.
The idea is, we want enough iterations of the loop
to feel confident of the results.
Now, by default, the test will run for about one second.
Which means it runs for a few seconds.
We're going to up it to about three seconds,
just to make sure we get enough iterations.
You gotta figure out what enough iterations are.
A million, a hundred million.
It's up to you.
So you can increase the bench time to get more iterations
to feel more confident, but again,
it's going to take longer to run.
So, look at this Benchmark function S print.
We've got our loop b.N.
We are calling S print.
I'm assigning the string coming back out, to s.
Which is a local variable.
We know that if we don't use the local variable,
Go's complier's going to complain.
I don't want to add extra CPU cycles here.
So, we assign it to a global variable, just so the compiler
won't throw it away.
That's all we're doing here with gs,
is to validate that we don't throw anything away.
That's not going to give us any extra cost
on this benchmark, because the real cost that we're
looking at here is inside the loop.
Okay, so, we got S print and we got S print f.
So, I want you to make a guess on which ones
going to be faster.
I'm guessing S print,
because we're not doing any formatting.
Again, this is in an underscore test file.
So, if I move back over to our
terminal, there it is.
Basic underscore test.
And we're going to use go test like we did before.
But since there are no functions that start with test,
I don't really want to waste the tools time
looking for them.
And if there were functions that started with test,
last thing I want is them to run.
By default, go test will always look for
and run test functions.
So, if we only want to run the benchmarks,
then we gotta give it a regular expression
where it won't find anything.
That's why I like using none.
None isn't special, just a string that I know there are
no test functions around.
So, we'll filter out all those test functions
even though there are none.
It doesn't matter, I still like to say none as practice.
Then, we're going to use the bench flag,
and I'm going to use dot for everything,
remember, it's regular expression.
So, run none tests, but all the benchmarks.
Let's increase the benchtime to three seconds.
So, that uses basic time formatting that we would get out
of the time package, three s.
You know, you can do your hours, you can do your minutes.
There it is three seconds.
So, we're going to go from one second to three seconds.
And I'm just looking at CPU right now,
but you know what?
Let me show you what else we can do.
I can also tell it to look at memory allocations
just by saying benchmem.
So if I do benchmem, I'm also going to tell it
report allocations.
Remember, an allocation is anytime a value escapes
the stack and goes into the heap.
So, I think we're good here.
Go test, run none, bench dot, bench time three seconds.
Bench the memory.
Okay, brilliant.
So, I'm running this now,
and then those that were on Darwin,
we know we're in a amd64 architecture.
S print, now, the first number here as it runs,
is showing us how many times we executed that code
inside the loop.
That was 50 million times over that three seconds.
If you really noticed,
that whole test took almost nine seconds.
Okay.
Three, at least three seconds for each,
it looks like almost four seconds for each.
So, we ran this 50,000 times.
We identified that S print ran at 68 nanoseconds
per operation.
With five bites allocated over one object.
So there's one object
on the heap worth five bites of memory.
Notice, however, that we were able to run S print f
100 million times.
Because it ran faster, at 53 nanoseconds per op,
with the same memory allocation.
In other words, our guess was wrong.
S print f is actually faster than S print
by almost, what, in this particular run,
we're looking at about 15 nanoseconds per op.
I mean, we're talking small.
But this is why I'm always going to tell you
we cannot guess.
Do not guess about performance.
Do not guess.
When you are writing code, we optimize for correctness,
readability, simplicity, integrity,
because that's all you can really know about
at the time you're writing the code.
If you write code for performance,
especially if it's new code, you are guessing,
you're going to be wrong.
Get something working,
we can use these benchmarks to validate.
So, now we know we probably would have used S print
in our code, when from a performance standpoint,
S print f would have been better.
I mean, 15 nanoseconds per op,
but that's my point is,
we don't want to guess about performance.
So, we've just seen how we can create a basic benchmark
and run that on the command line.