pingcap / failpoint Goto Github PK
View Code? Open in Web Editor NEWAn implementation of failpoints for Golang.
License: Apache License 2.0
An implementation of failpoints for Golang.
License: Apache License 2.0
For example, I may register a defer
block to disable failpoint to ensure that it's triggered no matter intermediate error occurs, while I'm going to disable the failpoint on the fly of the code flow early to switch behavior.
Then, one failpoint can be disabled twice and with current implementation, this causes failpoint: failpoint is disabled
error.
func foo() {
failpoint.Enable(...)
defer failpoint.Disable(...)
// ... error may occur in this block, thus I need a defer block above
failpoint.Disable(...)
// failpoint has been disabled, but it will be disable again when executing the defer block
}
I'd like to make failpoint.Disable
idempotent.
// mockStmtCommitError
// gofail: var mockStmtCommitError bool
// if mockStmtCommitError && mockStmtCommitErrorOnce {
// mockStmtCommitErrorOnce = false
// return kv.ErrRetryable
// }
gofail.Enable("github.com/pingcap/tidb/session/mockStmtCommitError", return(true)
)
第二次进来的时候会打印错误信息
I have the following setup (all in the same directory):
go.mod
module example.com/modv1
go 1.20
require (
github.com/pingcap/failpoint v0.0.0-20220801062533-2eaa32854a6c
github.com/stretchr/testify v1.8.4
)
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/pingcap/errors v0.11.4 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
samplestruct.go
package main
import (
"log"
"github.com/pingcap/failpoint"
)
type SampleStruct struct {
cnt int
}
func (s *SampleStruct) increment() {
if _, _err_ := failpoint.Eval(_curpkg_("testPanic")); _err_ == nil {
log.Printf("[SampleStruct] SUCCESS!!! Executed testPanic FP: The count for s is %v", s.cnt)
}
s.cnt++
}
func (s *SampleStruct) decrement() {
s.cnt--
}
func (s *SampleStruct) incrementFP() string {
log.Printf("[SampleStruct] Executing incrementFP with old cnt %v", s.cnt)
s.cnt++
log.Printf("[SampleStruct] Executing incrementFP with new cnt %v", s.cnt)
if _, _err_ := failpoint.Eval(_curpkg_("doubleIncrement")); _err_ == nil {
s.cnt++
log.Printf("[SampleStruct] SUCCESS!!! Executed doubleIncrement FP. The count for s is %v", s.cnt)
}
return "single"
}
main.go
package main
import (
"fmt"
"log"
"os"
)
func main() {
log.Printf(fmt.Sprintf("[main] This is main function being executed in an executable at path %v", os.Args[0]))
//os.Setenv("Go_FAILPOINTS", "main/doubleIncrement=return(true);main/testPanic=return(true)")
log.Printf(fmt.Sprintf("[main] After FP code in main function"))
sampleStruct := SampleStruct{cnt: 1}
log.Printf("[main] sampleStruct cnt before increment is %v", sampleStruct.cnt)
sampleStruct.increment()
log.Printf("[main] sampleStruct cnt after increment is %v", sampleStruct.cnt)
log.Printf("[main] sampleStruct cnt before double increment is %v", sampleStruct.cnt)
sampleStruct.incrementFP()
log.Printf("[main] sampleStruct cnt after double increment is %v", sampleStruct.cnt)
}
sample_test.go
package main
import (
"log"
"testing"
"os/exec"
"os"
"fmt"
"bytes"
"github.com/stretchr/testify/assert"
)
func TestFailpoint(t *testing.T) {
log.Printf("[TestFailpoint] Executing TestFailpoint")
mySampleStruct := SampleStruct{cnt: 2}
assert.Equal(t, mySampleStruct.cnt, 2)
// fpsToEnable := map[string]struct{}{
// "doubleIncrement": {},
// }
// cmd := exec.Command("export", "GO_FAILPOINTS='main/doubleIncrement=return(true);main/testPanic=return(true)'")
// if err := cmd.Run(); err != nil {
// log.Fatal(fmt.Sprintf("Encountered error while setting up the GO_FAILPOINTS environment variable. Error: %v", err))
// } else {
// log.Printf("Successfully set up the GO_FAILPOINTS environment variable")
// }
os.Setenv("GO_FAILPOINTS","'main/doubleIncrement=return(true);main/testPanic=return(true)'")
log.Printf("[TestFailpoint] Successfully set up the GO_FAILPOINTS environment variable")
cmd := exec.Command("./modv1")
var out bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &stderr
if err := cmd.Start(); err != nil {
log.Printf(fmt.Sprintf("[TestFailpoint] Error while starting modv1 executable. Error: %v", err))
log.Printf("[TestFailpoint] Command error is: " + stderr.String())
} else {
log.Printf("[TestFailpoint] Successfully started modv1 executable")
log.Printf("[TestFailpoint] Result: " + out.String() + ",Error: " + stderr.String())
}
if err := cmd.Wait(); err != nil {
log.Printf(fmt.Sprintf("[TestFailpoint] Error while waiting for modv1 executable. Error: %v", err))
log.Printf("[TestFailpoint] Command error is: " + stderr.String())
} else {
log.Printf("[TestFailpoint] Successfully waited for modv1 executable")
log.Printf("[TestFailpoint] Result: " + out.String() + ",Error: " + stderr.String())
}
//failpoint.Enable("doubleIncrement", "return(1)")
log.Printf("[TestFailpoint] Calling increment")
mySampleStruct.increment()
log.Printf("[TestFailpoint] Called increment")
log.Printf("[TestFailpoint] Calling incrementFP")
result := mySampleStruct.incrementFP()
log.Printf("[TestFailpoint] Called incrementFP")
assert.Equal(t, mySampleStruct.cnt, 4)
assert.Equal(t, result, "single")
os.Unsetenv("GO_FAILPOINTS")
log.Printf("[TestFailpoint] Successfully unset the GO_FAILPOINTS environment variable")
}
What did you expect to see?
When I run go build
and set the environment variable export GO_FAILPOINTS=main/doubleIncrement=return(true);main/testPanic=return(true)
and then execute the ./modv1
executable I see that the failpoints in samplestruct.go
are hit and the failpoint closure is executed. I expected same behavior when I run go test -v
i.e. when the TestFailpoint
test is executed.
What did you see instead?
I did not see the failpoints in samplestruct.go
getting executed while running the test TestFalipoint
. Is this expected behavior? Is the failpoint code executed only in the compiled binary modv1
and not during test execution even though the environment variable GO_FAILPOINTS
is kept around (and additionally also set/unset during) test execution?
Versions of the failpoint
$ ./failpoint-ctl -V
ReleaseVersion ed9079f
BuildTS 2023-09-01 02:53:58
GitHash ed9079f47f3761cbbeadaded49d48e3264c23427
GitBranch fp-testing
GoVersion go version go1.21.0 darwin/arm64
Is your feature request related to a problem? Please describe:
When writing a unit test, I want to know if a failpoint is evaluated and the evaluated value.
Describe the feature you'd like:
When enabling a failpoint, pass in a function callback func(val interface{})
. Afterwards, whenever the failpoint is evaluated, failpoint is responsible to call callback
.
Describe alternatives you've considered:
Allow user to register a function to replace term
. Every time the failpoint is evaluated, it calls the function to get the value.
Teachability, Documentation, Adoption, Optimization:
Please answer these questions before submitting your issue. Thanks!
package main
import (
"github.com/pingcap/failpoint"
)
const (
Marker = "failpoint-name"
)
func F1(name string) string {
failpoint.Inject(Marker, nil)
return "hello " + name
}
What did you expect to see?
This file can be correctly rewritten by failpoint-ctl
tool
What did you see instead?
rewrite failed.
/bin/sh: Rewrite: command not found
make: *** [failpoint-enable] Error 127
Versions of the failpoint
failpoint-ctl -V
):ReleaseVersion bf45ab2
BuildTS 2019-05-15 09:48:44
GitHash bf45ab2
GitBranch master
GoVersion go version go1.12 darwin/amd64
I cannot have any idea that how to use failpoint. I tried write demo like this:
var outerVar = "declare in outer scope"
err := failpoint.Enable("failpoint-name", "return")
if err != nil {
fmt.Println(err)
return
}
failpoint.Inject("failpoint-name", func(val failpoint.Value) {
fmt.Println("unit-test", val, outerVar)
})
There isn't have any output. How could I use it in the right way?
Is your feature request related to a problem? Please describe:
Although failpoint-ctl disable
will automatically apply diff when the original source is changed, this doesn't always work, and it would be frustrating to be stuck in a state where failpoint-ctl disable
cannot proceed and you've got some changes hard to be extracted.
Most editors supporting Go recognize the generated source indicator and warns when you try to edit the file, thus preventing the need to merge diffs in the first place.
Describe the feature you'd like:
In the source code generated by failpoint-ctl enable
(all of the rewritten *.go
, *__failpoint_binding__.go
, and *.go__failpoint_stash__
), insert this comment on first line:
// Code generated by failpoint DO NOT EDIT.
Remove this line after failpoint-ctl disable
.
Describe alternatives you've considered:
Teachability, Documentation, Adoption, Optimization:
Please answer these questions before submitting your issue. Thanks!
For example, we have a file named main.go:
package main
import (
"github.com/pingcap/failpoint"
)
func F1(name string) string {
failpoint.Inject("failpoint-name", nil)
return "hello " + name
}
func main() {
failpoint.Enable("failpoint-name", "panic")
F1("ttt")
}
After transfrom the code with failpoint-ctl enable
, the generated binding__failpoint_binding__.go
looks like this:
package main
import "reflect"
type __failpointBindingType struct {pkgpath string}
var __failpointBindingCache = &__failpointBindingType{}
func init() {
__failpointBindingCache.pkgpath = reflect.TypeOf(__failpointBindingType{}).PkgPath()
}
func _curpkg_(name string) string {
return __failpointBindingCache.pkgpath + "/" + name
}
You may notice that binding__failpoint_binding__.go
is also in package main
, that means binding__failpoint_binding__.go
could not be found by main.go
.
Run go run main.go
then, you will find the following error:
./main.go:8:17: undefined: _curpkg_
program panics due to failpoint
./main.go:8:17: undefined: _curpkg_
$ ./bin/failpoint-ctl -V
ReleaseVersion bf45ab2
BuildTS 2019-05-15 09:48:44
GitHash bf45ab2
GitBranch master
GoVersion go version go1.12 darwin/amd64
Please answer these questions before submitting your issue. Thanks!
f := func(g func() error) {}
f(func() error {
failpoint.Inject("with-in-a-clusure", func() {
failpoint.Return(errors.New("mock err"))
})
return nil
})
exec failpoint-ctl enable
, the failpoint.Inject
remains unchanged.
What did you expect to see?
What did you see instead?
Versions of the failpoint
the definition in the source code about Value's type in "github.com/pingcap/failpoint/failpoint.go"
// Value represents value that retrieved from failpoint terms.
// It can be used as following types:
// 1. val.(int) // GO_FAILPOINTS="failpoint-name=return(1)"
// 2. val.(string) // GO_FAILPOINTS="failpoint-name=return('1')"
// 3. val.(bool) // GO_FAILPOINTS="failpoint-name=return(true)"
Value interface{}
but in my code
failpoint.Inject("demoPanic", func(val failpoint.Value) {
fmt.Println(val)
fmt.Println(val.(string))
}
and start my progrom with cmd:
GO_FAILPOINTS="mypackagename/demoPanic=return('100')" ./beego-backend
An exception was thrown
[panic.go:522] Handler crashed with error interface conversion: failpoint.Value is bool, not string ···
Please answer these questions before submitting your issue. Thanks!
failpoint.List()
to obtain all the enabled failpoint keys. However, I'm also retrieving deprecated keys from the list.failpoint-ctl version (run failpoint-ctl -V
):
(paste failpoint-ctl version here)
ReleaseVersion None
BuildTS None
GitHash None
GitBranch None
GoVersion None
Is your feature request related to a problem? Please describe:
Yes
Describe the feature you'd like:
Release tags and binary for failpoint.
Describe alternatives you've considered:
Create git tags and release binary for both linux and mac builds? Its easier for download the builds directly from github and include in CI/CD pipelines as apposed to clone the entire repo and make builds.
Teachability, Documentation, Adoption, Optimization:
Users wanting to include failopint in their build pipelines , could directly curl/wget the tar files from github and use failpoint-ctl enable
in their repository. Its much simpler that way , instead of cloning the entire repo and making builds manually.
The REDAME file teaches me how to use failpoint when staring a program.
However, it does not tell about injecting faults to a program at running time.
For example,
First I start a go program :
./test
After 5 mins, I set the ENV GO_FAILPOINTS="main/testPanic=return(true)"
The fault cannot be injected to the program.
How can I inject faults to a program after running the progarm?
Lines 272 to 278 in 7acb0f0
There seems no reason we do it. Either return an error or log at a proper level is reasonable.
Please answer these questions before submitting your issue. Thanks!
package testpkg
import (
"fmt"
"github.com/pingcap/failpoint"
)
func test() {
func() {
failpoint.Inject("testPoint", func() {
fmt.Println()
})
}()
}
then failpoint-ctl enable
~
func test() {
func() {
if _, ok := failpoint.Eval(_curpkg_("testPoint")); ok {
fmt.Println()
}
}()
}
func test() {
func() {
failpoint.Inject("testPoint", func() {
fmt.Println()
})
}()
}
failpoint-ctl version (run failpoint-ctl -V
):
ReleaseVersion None
BuildTS None
GitHash None
GitBranch None
GoVersion None
newest tidb master
Please answer these questions before submitting your issue. Thanks!
f := func(g func() error) {}
f(func() error {
failpoint.Inject("XXX", func() {
failpoint.Return(errors.New("mock error"))
})
return nil
})
What did you expect to see?
After enabled failpoint,failpoint.Inject
can be changed to failpoint.Eval
.
What did you see instead?
Nothing changed
Versions of the failpoint
failpoint-ctl -V
):v0.0.0-20200702092429-9f69995143ce
go get github.com/pingcap/failpoint/
cd failpoint
go test .
/V/d/g/s/g/p/failpoint (master|✔) [2] $ go test .
failed to parse "invalid" past "invalid"
failed to enable "failpoint-name=invalid" (failpoint: could not parse terms)
----------------------------------------------------------------------
FAIL: http_test.go:47: httpSuite.TestServeHTTP
http_test.go:124:
c.Assert(err, IsNil)
... value *url.Error = &url.Error{Op:"Get", URL:"http://127.0.0.1:23389/failpoint-env1", Err:(*net.OpError)(0xc000172050)} ("Get http://127.0.0.1:23389/failpoint-env1: dial tcp 127.0.0.1:23389: connect: connection refused")
failed to parse "invalid" past "invalid"
failed to enable "runtime-test-2=invalid" (failpoint: could not parse terms)
----------------------------------------------------------------------
FAIL: runtime_test.go:21: runtimeSuite.TestRuntime
runtime_test.go:133:
c.Assert(ok, IsTrue)
... obtained bool = false
OOPS: 1 passed, 2 FAILED
--- FAIL: TestFailpoint (1.00s)
failed to parse "invalid" past "invalid"
failed to enable "failpoint-name=invalid" (failpoint: could not parse terms)
----------------------------------------------------------------------
FAIL: http_test.go:47: httpSuite.TestServeHTTP
http_test.go:124:
c.Assert(err, IsNil)
... value *url.Error = &url.Error{Op:"Get", URL:"http://127.0.0.1:23389/failpoint-env1", Err:(*net.OpError)(0xc000172d70)} ("Get http://127.0.0.1:23389/failpoint-env1: dial tcp 127.0.0.1:23389: connect: connection refused")
failed to parse "invalid" past "invalid"
failed to enable "runtime-test-2=invalid" (failpoint: could not parse terms)
----------------------------------------------------------------------
FAIL: runtime_test.go:21: runtimeSuite.TestRuntime
runtime_test.go:133:
c.Assert(ok, IsTrue)
... obtained bool = false
OOPS: 1 passed, 2 FAILED
--- FAIL: TestHttp (1.00s)
failed to parse "invalid" past "invalid"
failed to enable "failpoint-name=invalid" (failpoint: could not parse terms)
----------------------------------------------------------------------
FAIL: http_test.go:47: httpSuite.TestServeHTTP
http_test.go:124:
c.Assert(err, IsNil)
... value *url.Error = &url.Error{Op:"Get", URL:"http://127.0.0.1:23389/failpoint-env1", Err:(*net.OpError)(0xc000146500)} ("Get http://127.0.0.1:23389/failpoint-env1: dial tcp 127.0.0.1:23389: connect: connection refused")
failed to parse "invalid" past "invalid"
failed to enable "runtime-test-2=invalid" (failpoint: could not parse terms)
----------------------------------------------------------------------
FAIL: runtime_test.go:21: runtimeSuite.TestRuntime
runtime_test.go:133:
c.Assert(ok, IsTrue)
... obtained bool = false
OOPS: 1 passed, 2 FAILED
--- FAIL: TestNewRestorer (1.01s)
FAIL
FAIL github.com/pingcap/failpoint 3.030s
/V/d/g/s/g/p/failpoint (master|✔) [1] $
I write this in foo.go:
func pingCapFail() (string, failpoint.Value) {
failpoint.Inject("failpoint-name", func(val failpoint.Value) {
failpoint.Return("unit-test", val)
})
return "success", nil
}
and i run failpoint-ctl enable
, code becomes this;
func pingCapFail() (string, failpoint.Value) {
if val, ok := failpoint.Eval(_curpkg_("failpoint-name")); ok {
return "unit-test", val
}
return "success", nil
}
then in foo_test.go ,i write my test code like this:
func Test_pingCapFail(t *testing.T) {
failpoint.Enable("failpoint-name", "return(5)")
got, got1 := pingCapFail()
if got != "unit-test" {
t.Errorf("pingCapFail() got = %v, want %v", got, "unit-test")
}
if !reflect.DeepEqual(got1, 5) {
t.Errorf("pingCapFail() got1 = %v, want %v", got1, 5)
}
}
and it fails, and obviously the failpoint is not activated, can someone tell me why?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.