testing: clarification needed for testing.F method usage rules

[Yakog-ux]

Go version

go version go1.24.2 linux/amd64

Output of go env in your module/workspace:

/

What did you do?

The documentation for the testing.F type currently states:

*F methods can only be called before (*F).Fuzz. Once the test is executing the fuzz target, only (*T) methods can be used.

Thus, two requirements are:

1. methods of the F type can only be used/called before (*F).Fuzz, and
2. within a fuzz target, only methods of type T are allowed to be called.

In the following example, the second requirement is violated:

func FuzzFoo(f *testing.F) {
	f.Add(5)
	f.Fuzz(func(t *testing.T, a int) {
		t.Logf("%v\n", a)
		f.Skip() // fuzz target must not call any *F methods
	})
}

Since the 2. requirement is violated, the test fails (as expected) and the following is printed:

pk1/test2_test.go:18:3: fuzz target must not call any *F methods

Next, the following code violates the first requirement:

func FuzzFoo(f *testing.F) {
	f.Add(5)
	f.Fuzz(func(t *testing.T, a int) {
		t.Logf("%v\n", a)
	})
	f.Cleanup(func() {
		f.Logf("smth")
	})

	f.Logf("end")
}

However, in this case no error occurs ("smth" and "end" are printed, the test passed).

This raises confusion: is the statement in the documentation meant to be a strict requirement (i.e., calling any F method after (*F).Fuzz is forbidden, as in the case of the 2. requirement), or is it just a recommendation for correct setup before fuzzing starts?

Based on what is written, it seems like a requirement. However, based on the behavior, it seems like it is just a suggestion.

If it is merely a recommendation, perhaps the documentation could be updated to make that clearer — for example, by stating that calling methods after (*F).Fuzz may result in undefined behavior, instead of suggesting it is completely disallowed.

Additionally, documentation for the (*F).Add states the following:

This will be a no-op if called after or within the fuzz target, and args must match the arguments for the fuzz target.

So, in the type description it says that it is not allowed to be used after a fuzz target, and here it says that it is, but it is a no-op.

Also, in my opinion it should just say "This will be a no-op if called after the fuzz target" (assuming that requirement 1 is just a suggestion; not requirement). Calling inside the fuzz target violates requirement 2 and the test will fail (so it is not a no-op).

What did you see happen?

Inconsistency between documentation and behavior.

What did you expect to see?

Consistency between documentation and behavior.

[gabyhelp]

Related Issues

• testing: enforce the documented restriction on fuzz target result parameters #51222 (closed)
• testing: prevent (*testing.F) methods from being called inside (*testing.F).Fuzz fn #48988 (closed)

Related Documentation

• Package testing > type F ¶ 1.18 > func (*F) Fuzz ¶ 1.18
• Go Fuzzing > Writing fuzz tests > Requirements
• Package testing > type F ¶ 1.18

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

Even if it doesn't break, I don't think we'd want to weaken it. It is poor style anyway: cleanup, like defer, is generally registered first.

The documentation for f.Add reinforces the point: don't try to use f inside the fuzz target, it won't work as you intend.

[Yakog-ux]

@seankhliao

Even if it doesn't break, I don't think we'd want to weaken it. It is poor style anyway: cleanup, like defer, is generally registered first.

It's not just Cleanup, it's about all methods. I just used Cleanup as an example.

The documentation for f.Add reinforces the point: don't try to use f inside the fuzz target, it won't work as you intend.

It says that calling inside a fuzz target is a no-op, even though it is not (it is not allowed, the test will fail).

What is even more strange is that the third sentence in the observed paragraph (related to the description of the F type) says:

The only *F methods that are allowed in the (*F).Fuzz function are (*F).Failed and (*F).Name.

and just two sentences earlier it was written that methods of type F can only be called before (*F).Fuzz. How can they be allowed, if it was previously written that method calls are only allowed before (*F).Fuzz?