SetFinalizer
runtime.SetFinalizer:
1
|
func SetFinalizer(obj interface{}, finalizer interface{})
|
对象可以关联一个SetFinalizer函数, 当gc检测到unreachable对象有关联的SetFinalizer函数时,会执行关联的SetFinalizer函数, 同时取消关联。 这样当下一次gc的时候,对象重新处于unreachable状态并且没有SetFinalizer关联, 就会被回收。
1
2
3
4
5
6
7
8
9
|
func main() {
// i 就是后面说的 数据对象
var i = 3
// 这里的func 就是后面一直说的 finalizer
runtime.SetFinalizer(&i, func(i *int) {
fmt.Println(i, *i, "set finalizer")
})
time.Sleep(time.Second * 5)
}
|
SetFinalizer将x的终止器设置为f。当垃圾收集器发现一个不能接触的(即引用计数为零,程序中不能再直接或间接访问该对象)具有终止器的块时,它会清理该关联(对象到终止器)并在独立go程调用f(x)。这使x再次可以接触,但没有了绑定的终止器。如果SetFinalizer没有被再次调用,下一次垃圾收集器将视x为不可接触的,并释放x。
SetFinalizer(x, nil)
会清理任何绑定到x的终止器。
参数x必须是一个指向通过new申请的对象的指针,或者通过对复合字面值取址得到的指针。参数f必须是一个函数,它接受单个可以直接用x类型值赋值的参数,也可以有任意个被忽略的返回值。如果这两条任一条不被满足,本函数就会中断程序。
终止器会按依赖顺序执行:如果A指向B,两者都有终止器,且它们无法从其它方面接触,只有A的终止器执行;A被释放后,B的终止器就可以执行。如果一个循环结构包含一个具有终止器的块,该循环不能保证会被当垃圾收集,终止器也不能保证会执行;因为没有尊重依赖关系的顺序。
仔细看文档,还有几个需要注意的点:
-
即使程序正常结束或者发生错误, 但是在对象被 gc 选中并被回收之前,SetFinalizer 都不会执行, 所以不要在SetFinalizer中执行将内存中的内容flush到磁盘这种操作
-
SetFinalizer 最大的问题是延长了对象生命周期。在第一次回收时执行 Finalizer 函数,且目标对象重新变成可达状态,直到第二次才真正 “销毁”。这对于有大量对象分配的高并发算法,可能会造成很大麻烦
-
指针构成的 “循环引⽤” 加上 runtime.SetFinalizer
会导致内存泄露
无保障性
x的终止器会在x变为不可接触之后的任意时间被调度执行。不保证终止器会在程序退出前执行,因此一般终止器只用于在长期运行的程序中释放关联到某对象的非内存资源。例如,当一个程序丢弃一个os.File
对象时没有调用其Close方法,该os.File对象可以使用终止器去关闭对应的操作系统文件描述符。但依靠终止器去刷新内存中的I/O缓冲如bufio.Writer是错误的,因为缓冲不会在程序退出时被刷新。
如果*x
的大小为0字节,不保证终止器会执行。
举一个使用了 Finalizer 的例子
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
package main
import (
"fmt"
"math/rand"
"runtime"
"strconv"
"time"
)
type Foo struct {
a int
}
func main() {
for i := 0; i < 3; i++ {
f := NewFoo(i)
println(f.a)
}
runtime.GC()
}
//go:noinline
func NewFoo(i int) *Foo {
f := &Foo{a: rand.Intn(50)}
runtime.SetFinalizer(f, func(f*Foo) {
fmt.Println(`foo` + strconv.Itoa(i) + `has been garbage collected`)
})
return f
}
|
这段程序将会在这个循环中创建三个 struct 的的实例,并将每个实例都绑定一个 finalizer。之后垃圾回收器将会被调用,并回收之前创建的实例。运行这个程序,将会给到我们如下输出:
在程序无法获取到一个 obj 所指向的对象后的任意时刻,finalizer 被调度运行,且无法保证 finalizer 运行在程序退出之前。因此一般情况下,因此它们仅用于在长时间运行的程序上释放一些与对象关联的非内存资源。
在调用 finalizer 之前,runtime 不提供有关延迟的任何保证。让我们试着去修改我们的程序,通过在调用垃圾回收器之后添加一个一秒的 sleep:
1
2
3
4
5
|
31
37
47
foo 1 has been garbage collected
foo 0 has been garbage collected
|
现在我们的 finalizer 已经被调用了,然而,它们其中一个消失了。我们的 finalizers 与垃圾回收器相连接,并且垃圾回收器回收以及清理数据的方式将会对 finalizers 的调用产生影响。
工作流
之前的例子可能让我认为 Go 仅在释放我们所定义的 struct 的内存之前调用 finalizers。
让我们深入其中,看看在更多的 Allocation 中到底发生了些什么。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
|
package main
import (
"fmt"
"math/rand"
"runtime"
"runtime/debug"
"strconv"
"time"
)
type Foo struct {
a int
}
func main() {
debug.SetGCPercent(-1)
var ms runtime.MemStats
runtime.ReadMemStats(&ms)
fmt.Printf("Allocation: %f Mb, Number of allocation: %d\n", float32(ms.HeapAlloc)/float32(1024*1204), ms.HeapObjects)
for i := 0; i < 1000000; i++ {
f := NewFoo(i)
_ = fmt.Sprintf("%d", f.a)
}
runtime.ReadMemStats(&ms)
fmt.Printf("Allocation: %f Mb, Number of allocation: %d\n", float32(ms.HeapAlloc)/float32(1024*1204), ms.HeapObjects)
runtime.GC()
time.Sleep(time.Second)
runtime.ReadMemStats(&ms)
fmt.Printf("Allocation: %f Mb, Number of allocation: %d\n", float32(ms.HeapAlloc)/float32(1024*1204), ms.HeapObjects)
runtime.GC()
time.Sleep(time.Second)
}
//go:noinline
func NewFoo(i int) *Foo {
f := &Foo{a: rand.Intn(50)}
runtime.SetFinalizer(f, func(f*Foo) {
_ = fmt.Sprintf("foo " + strconv.Itoa(i) + " has been garbage collected")
})
return f
}
|
一百万个 structs 和 finalizers 被创建出来,下面是输出:
1
2
3
|
Allocation: 0.090862 Mb, Number of allocation: 137
Allocation: 31.107506 Mb, Number of allocation: 2390078
Allocation: 110.052666 Mb, Number of allocation: 4472742
|
让我们再试一次,这次不用 finalizers:
1
2
3
|
Allocation: 0.090694 Mb, Number of allocation: 136
Allocation: 18.129814 Mb, Number of allocation: 1390078
Allocation: 0.094451 Mb, Number of allocation: 154
|
看起来没有任何资源在内存中被清理掉,即使垃圾回收器被触发,且 finalizers 也运行。为了理解这一行为,让我们回到那篇关于 runtime 的文档:
当垃圾回收器发现了一个已关联 finalizer 的无法访问的块,这说明了关联操作与运行 finalizer 是在一个单独的 gorountine 下。这让 obj 再次可访问,不过现在没有了一个关联的 finalizer,假设 SetFinalizer 没有再次被调用,当下次垃圾回收器看到这个 obj 时,它是不可被访问的,并将回收它。
如我们所见,finalizers 首先会被移除,然后内存将在下一次循环中被释放,让我们再次运行第一个例子,并加上两个强制的垃圾回收操作。
1
2
3
4
|
Allocation: 0.090862 Mb, Number of allocation: 137
Allocation: 31.107506 Mb, Number of allocation: 2390078
Allocation: 110.052666 Mb, Number of allocation: 4472742
Allocation: 0.099220 Mb, Number of allocation: 166
|
我们可以清楚地看到,第二次运行将会清理数据,finalizers 最终也对性能和内存使用产生了轻微的作用。
性能表现
下文阐述了为何 finalizers 逐个运行:
一个单独 Goroutine 为了一个程序运行了所有的 finalizers,然而,如果一个 finalizer 必须长时间运行,则需要开启一个新的 gorountine。
仅一个 Goroutine 将会运行 finalizers,并且任何超重任务都需要开启一个新的 gorountine。当 finalizers 运行时,垃圾回收器并没有停止且并发运行中。因此 finalizer 并不该影响你的应用的性能表现。
同时,一旦 finalizer 不再被需要,Go 提供了一个方法来移除它。
1
|
runtime.SetFinalizer(p, nil)
|
它允许我们根据使用情况动态地移除 finalizers。
结构概览
heap:
1
2
3
4
5
6
|
type mspan struct {
...
// 当前span上所有对象的special串成链表
// special中有个offset,就是数据对象在span上的offset,通过offset,将数据对象和special关联起来
specials *special // linked list of special records sorted by offset.
}
|
special:
1
2
3
4
5
6
7
|
//go:notinheap
type special struct {
next *special // linked list in span
// 数据对象在span上的offset
offset uint16 // span offset of object
kind byte // kind of special
}
|
specialfinalizer:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
// The described object has a finalizer set for it.
//
// specialfinalizer is allocated from non-GC'd memory, so any heap
// pointers must be specially handled.
//
//go:notinheap
type specialfinalizer struct {
special special
fn *funcval // May be a heap pointer.
// return的数据的大小
nret uintptr
// 第一个参数的类型
fint *_type // May be a heap pointer, but always live.
// 与finalizer关联的数据对象的指针类型
ot *ptrtype // May be a heap pointer, but always live.
}
|
finalizer:
1
2
3
4
5
6
7
8
|
// NOTE: Layout known to queuefinalizer.
type finalizer struct {
fn *funcval // function to call (may be a heap pointer)
arg unsafe.Pointer // ptr to object (may be a heap pointer)
nret uintptr // bytes of return values from fn
fint *_type // type of first argument of fn
ot *ptrtype // type of ptr to object (may be a heap pointer)
}
|
全局变量
1
2
3
4
5
6
7
8
9
10
11
12
|
var finlock mutex // protects the following variables
// 运行finalizer的g,只有一个g,不用的时候休眠,需要的时候再唤醒
var fing*g // goroutine that runs finalizers
// finalizer的全局队列,这里是已经设置的finalizer串成的链表
var finq*finblock // list of finalizers that are to be executed
// 已经释放的finblock的链表,用finc缓存起来,以后需要使用的时候可以直接取走,避免再走一遍内存分配了
var finc*finblock // cache of free blocks
var finptrmask [_FinBlockSize / sys.PtrSize / 8]byte
var fingwait bool // fing的标志位,通过 fingwait和fingwake,来确定是否需要唤醒fing
var fingwake bool
// 所有的blocks串成的链表
var allfin*finblock // list of all blocks
|
SetFinalizer
根据 数据对象 ,生成一个special
对象,并绑定到 数据对象 所在的span,串联到span.specials
上,并且确保fing的存在
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
|
// SetFinalizer sets the finalizer associated with obj to the provided
// finalizer function. When the garbage collector finds an unreachable block
// with an associated finalizer, it clears the association and runs
// finalizer(obj) in a separate goroutine. This makes obj reachable again,
// but now without an associated finalizer. Assuming that SetFinalizer
// is not called again, the next time the garbage collector sees
// that obj is unreachable, it will free obj.
//
// SetFinalizer(obj, nil) clears any finalizer associated with obj.
//
// The argument obj must be a pointer to an object allocated by calling
// new, by taking the address of a composite literal, or by taking the
// address of a local variable.
// The argument finalizer must be a function that takes a single argument
// to which obj's type can be assigned, and can have arbitrary ignored return
// values. If either of these is not true, SetFinalizer may abort the
// program.
//
// Finalizers are run in dependency order: if A points at B, both have
// finalizers, and they are otherwise unreachable, only the finalizer
// for A runs; once A is freed, the finalizer for B can run.
// If a cyclic structure includes a block with a finalizer, that
// cycle is not guaranteed to be garbage collected and the finalizer
// is not guaranteed to run, because there is no ordering that
// respects the dependencies.
//
// The finalizer is scheduled to run at some arbitrary time after the
// program can no longer reach the object to which obj points.
// There is no guarantee that finalizers will run before a program exits,
// so typically they are useful only for releasing non-memory resources
// associated with an object during a long-running program.
// For example, an os.File object could use a finalizer to close the
// associated operating system file descriptor when a program discards
// an os.File without calling Close, but it would be a mistake
// to depend on a finalizer to flush an in-memory I/O buffer such as a
// bufio.Writer, because the buffer would not be flushed at program exit.
//
// It is not guaranteed that a finalizer will run if the size of *obj is
// zero bytes.
//
// It is not guaranteed that a finalizer will run for objects allocated
// in initializers for package-level variables. Such objects may be
// linker-allocated, not heap-allocated.
//
// A finalizer may run as soon as an object becomes unreachable.
// In order to use finalizers correctly, the program must ensure that
// the object is reachable until it is no longer required.
// Objects stored in global variables, or that can be found by tracing
// pointers from a global variable, are reachable. For other objects,
// pass the object to a call of the KeepAlive function to mark the
// last point in the function where the object must be reachable.
//
// For example, if p points to a struct, such as os.File, that contains
// a file descriptor d, and p has a finalizer that closes that file
// descriptor, and if the last use of p in a function is a call to
// syscall.Write(p.d, buf, size), then p may be unreachable as soon as
// the program enters syscall.Write. The finalizer may run at that moment,
// closing p.d, causing syscall.Write to fail because it is writing to
// a closed file descriptor (or, worse, to an entirely different
// file descriptor opened by a different goroutine). To avoid this problem,
// call runtime.KeepAlive(p) after the call to syscall.Write.
//
// A single goroutine runs all finalizers for a program, sequentially.
// If a finalizer must run for a long time, it should do so by starting
// a new goroutine.
func SetFinalizer(obj interface{}, finalizer interface{}) {
if debug.sbrk != 0 {
// debug.sbrk never frees memory, so no finalizers run
// (and we don't have the data structures to record them).
return
}
e := efaceOf(&obj)
etyp := e._type
if etyp == nil {
throw("runtime.SetFinalizer: first argument is nil")
}
if etyp.kind&kindMask != kindPtr {
throw("runtime.SetFinalizer: first argument is " + etyp.string() + ", not pointer")
}
// ---- 省略数据校验的逻辑 ---
ot := (*ptrtype)(unsafe.Pointer(etyp))
if ot.elem == nil {
throw("nil elem type!")
}
// find the containing object
// 在内存中找不到分配的地址时 base==0,setFinalizer 是在内存回收的时候调用,没有分配就不会回收
base, _, _ := findObject(uintptr(e.data), 0, 0)
if base == 0 {
// 0-length objects are okay.
if e.data == unsafe.Pointer(&zerobase) {
return
}
// Global initializers might be linker-allocated.
// var Foo = &Object{}
// func main() {
// runtime.SetFinalizer(Foo, nil)
// }
// The relevant segments are: noptrdata, data, bss, noptrbss.
// We cannot assume they are in any order or even contiguous,
// due to external linking.
for datap := &firstmoduledata; datap != nil; datap = datap.next {
if datap.noptrdata <= uintptr(e.data) && uintptr(e.data) < datap.enoptrdata ||
datap.data <= uintptr(e.data) && uintptr(e.data) < datap.edata ||
datap.bss <= uintptr(e.data) && uintptr(e.data) < datap.ebss ||
datap.noptrbss <= uintptr(e.data) && uintptr(e.data) < datap.enoptrbss {
return
}
}
throw("runtime.SetFinalizer: pointer not in allocated block")
}
if uintptr(e.data) != base {
// As an implementation detail we allow to set finalizers for an inner byte
// of an object if it could come from tiny alloc (see mallocgc for details).
if ot.elem == nil || ot.elem.ptrdata != 0 || ot.elem.size >= maxTinySize {
throw("runtime.SetFinalizer: pointer not at beginning of allocated block")
}
}
f := efaceOf(&finalizer)
ftyp := f._type
// 如果 finalizer type == nil,尝试移除(没有的话,就不需要移除了)
if ftyp == nil {
// switch to system stack and remove finalizer
systemstack(func() {
removefinalizer(e.data)
})
return
}
// --- 对finalizer参数数量及类型进行校验 --
if ftyp.kind&kindMask != kindFunc {
throw("runtime.SetFinalizer: second argument is " + ftyp.string() + ", not a function")
}
ft := (*functype)(unsafe.Pointer(ftyp))
if ft.dotdotdot() {
throw("runtime.SetFinalizer: cannot pass " + etyp.string() + " to finalizer " + ftyp.string() + " because dotdotdot")
}
if ft.inCount != 1 {
throw("runtime.SetFinalizer: cannot pass " + etyp.string() + " to finalizer " + ftyp.string())
}
fint := ft.in()[0]
switch {
case fint == etyp:
// ok - same type
goto okarg
case fint.kind&kindMask == kindPtr:
if (fint.uncommon() == nil || etyp.uncommon() == nil) && (*ptrtype)(unsafe.Pointer(fint)).elem == ot.elem {
// ok - not same type, but both pointers,
// one or the other is unnamed, and same element type, so assignable.
goto okarg
}
case fint.kind&kindMask == kindInterface:
ityp := (*interfacetype)(unsafe.Pointer(fint))
if len(ityp.mhdr) == 0 {
// ok - satisfies empty interface
goto okarg
}
if iface := assertE2I2(ityp, *efaceOf(&obj)); iface.tab != nil {
goto okarg
}
}
throw("runtime.SetFinalizer: cannot pass " + etyp.string() + " to finalizer " + ftyp.string())
okarg:
// compute size needed for return parameters
// 计算返回参数的大小并进行对齐
nret := uintptr(0)
for _, t := range ft.out() {
nret = alignUp(nret, uintptr(t.align)) + uintptr(t.size)
}
nret = alignUp(nret, sys.PtrSize)
// make sure we have a finalizer goroutine
// 确保 finalizer 有一个 goroutine
createfing()
systemstack(func() {
// 切换到g0,添加finalizer,并且不能重复设置
if !addfinalizer(e.data, (*funcval)(f.data), nret, fint, ot) {
throw("runtime.SetFinalizer: finalizer already set")
}
})
}
|
这里逻辑没什么复杂的,只是在参数、类型的判断等上面,比较的麻烦
removefinalizer
通过removespecial,找到数据对象p所对应的special对象,如果找到的话,释放mheap上对应的内存
1
2
3
4
5
6
7
8
9
10
11
12
|
// Removes the finalizer (if any) from the object p.
func removefinalizer(p unsafe.Pointer) {
// 根据数据p找到对应的special对象
s := (*specialfinalizer)(unsafe.Pointer(removespecial(p, _KindSpecialFinalizer)))
if s == nil {
return // there wasn't a finalizer to remove
}
lock(&mheap_.speciallock)
// 释放找到的special所对应的内存
mheap_.specialfinalizeralloc.free(unsafe.Pointer(s))
unlock(&mheap_.speciallock)
}
|
这里的函数,虽然叫removefinalizer, 但是这里暂时跟finalizer结构体没有关系,都是在跟special结构体打交道,后面的addfinalizer也是一样的
removespecial遍历数据所在的span的specials,如果找到了指定数据p的special的话,就从specials中移除,并返回
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
|
// Removes the Special record of the given kind for the object p.
// Returns the record if the record existed, nil otherwise.
// The caller must FixAlloc_Free the result.
func removespecial(p unsafe.Pointer, kind uint8) *special {
// 找到数据p所在的span
span := spanOfHeap(uintptr(p))
if span == nil {
throw("removespecial on invalid pointer")
}
// Ensure that the span is swept.
// Sweeping accesses the specials list w/o locks, so we have
// to synchronize with it. And it's just much safer.
mp := acquirem()
// 保证span被清扫过了
span.ensureSwept()
// 获取数据p的偏移量,根据偏移量去寻找p对应的special
offset := uintptr(p) - span.base()
var result *special
lock(&span.speciallock)
t := &span.specials
// 遍历span.specials这个链表
for {
s := *t
if s == nil {
break
}
// This function is used for finalizers only, so we don't check for
// "interior" specials (p must be exactly equal to s->offset).
if offset == uintptr(s.offset) && kind == s.kind {
// 找到了,修改指针,将当前找到的special移除
*t = s.next
result = s
break
}
t = &s.next
}
if span.specials == nil {
spanHasNoSpecials(span)
}
unlock(&span.speciallock)
releasem(mp)
return result
}
|
addfinalizer
正好跟removefinalizer相反,这个就是根据数据对象p,创建对应的special,然后添加到span.specials链表上面
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
// Adds a finalizer to the object p. Returns true if it succeeded.
func addfinalizer(p unsafe.Pointer, f *funcval, nret uintptr, fint *_type, ot *ptrtype) bool {
lock(&mheap_.speciallock)
// 分配出来一块内存供finalizer使用
s := (*specialfinalizer)(mheap_.specialfinalizeralloc.alloc())
unlock(&mheap_.speciallock)
s.special.kind = _KindSpecialFinalizer
s.fn = f
s.nret = nret
s.fint = fint
s.ot = ot
if addspecial(p, &s.special) {
// This is responsible for maintaining the same
// GC-related invariants as markrootSpans in any
// situation where it's possible that markrootSpans
// has already run but mark termination hasn't yet.
if gcphase != _GCoff {
base, _, _ := findObject(uintptr(p), 0, 0)
mp := acquirem()
gcw := &mp.p.ptr().gcw
// Mark everything reachable from the object
// so it's retained for the finalizer.
scanobject(base, gcw)
// Mark the finalizer itself, since the
// special isn't part of the GC'd heap.
scanblock(uintptr(unsafe.Pointer(&s.fn)), sys.PtrSize, &oneptrmask[0], gcw, nil)
releasem(mp)
}
return true
}
// There was an old finalizer
// 没有添加成功,是因为p已经有了一个special对象了
lock(&mheap_.speciallock)
mheap_.specialfinalizeralloc.free(unsafe.Pointer(s))
unlock(&mheap_.speciallock)
return false
}
|
addspecial添加special的主逻辑
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
|
// Adds the special record s to the list of special records for
// the object p. All fields of s should be filled in except for
// offset & next, which this routine will fill in.
// Returns true if the special was successfully added, false otherwise.
// (The add will fail only if a record with the same p and s->kind
// already exists.)
func addspecial(p unsafe.Pointer, s *special) bool {
span := spanOfHeap(uintptr(p))
if span == nil {
throw("addspecial on invalid pointer")
}
// Ensure that the span is swept.
// Sweeping accesses the specials list w/o locks, so we have
// to synchronize with it. And it's just much safer.
// 同 removerspecial一样,确保这个span已经清扫过了
mp := acquirem()
span.ensureSwept()
offset := uintptr(p) - span.base()
kind := s.kind
lock(&span.speciallock)
// Find splice point, check for existing record.
t := &span.specials
for {
x := *t
if x == nil {
break
}
if offset == uintptr(x.offset) && kind == x.kind {
// 已经存在了,不能在增加了,一个数据对象,只能绑定一个finalizer
unlock(&span.speciallock)
releasem(mp)
return false // already exists
}
if offset < uintptr(x.offset) || (offset == uintptr(x.offset) && kind < x.kind) {
break
}
t = &x.next
}
// Splice in record, fill in offset.
// 添加到 specials 队列尾
s.offset = uint16(offset)
s.next = *t
*t = s
spanHasSpecials(span)
unlock(&span.speciallock)
releasem(mp)
return true
}
|
createfing
这个函数是保证,创建了finalizer之后,有一个goroutine去运行,这里只运行一次,这个goroutine会由全局变量 fing 记录
1
2
3
4
5
6
7
8
|
func createfing() {
// 创建一个goroutine,时刻监控运行
// start the finalizer goroutine exactly once
if fingCreate == 0 && atomic.Cas(&fingCreate, 0, 1) {
// 开启一个goroutine运行
go runfinq()
}
}
|
接下来就分析一下执行流程runfinq吧
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
|
// This is the goroutine that runs all of the finalizers
func runfinq() {
var (
frame unsafe.Pointer
framecap uintptr
argRegs int
)
for {
lock(&finlock)
// 获取finq 全局队列,并清空全局队列
fb := finq
finq = nil
if fb == nil {
// 如果全局队列为空,休眠当前g,等待被唤醒
gp := getg()
fing = gp
// 设置fing的状态标志位
fingwait = true
goparkunlock(&finlock, waitReasonFinalizerWait, traceEvGoBlock, 1)
continue
}
argRegs = intArgRegs
unlock(&finlock)
if raceenabled {
racefingo()
}
// 循环执行runq链表里的fin数组
for fb != nil {
for i := fb.cnt; i > 0; i-- {
f := &fb.fin[i-1]
var regs abi.RegArgs
var framesz uintptr
if argRegs > 0 {
// The args can always be passed in registers if they're
// available, because platforms we support always have no
// argument registers available, or more than 2.
//
// But unfortunately because we can have an arbitrary
// amount of returns and it would be complex to try and
// figure out how many of those can get passed in registers,
// just conservatively assume none of them do.
framesz = f.nret
} else {
// Need to pass arguments on the stack too.
// 获取存储当前finalizer的返回数据的大小,如果比之前大,则分配
framesz = unsafe.Sizeof((interface{})(nil)) + f.nret
}
if framecap < framesz {
// The frame does not contain pointers interesting for GC,
// all not yet finalized objects are stored in finq.
// If we do not mark it as FlagNoScan,
// the last finalized object is not collected.
frame = mallocgc(framesz, nil, true)
framecap = framesz
}
if f.fint == nil {
throw("missing type in runfinq")
}
r := frame
if argRegs > 0 {
r = unsafe.Pointer(®s.Ints)
} else {
// frame is effectively uninitialized
// memory. That means we have to clear
// it before writing to it to avoid
// confusing the write barrier.
// 清空frame内存存储
*(*[2]uintptr)(frame) = [2]uintptr{}
}
switch f.fint.kind & kindMask {
case kindPtr:
// direct use of pointer
*(*unsafe.Pointer)(r) = f.arg
case kindInterface:
ityp := (*interfacetype)(unsafe.Pointer(f.fint))
// set up with empty interface
(*eface)(r)._type = &f.ot.typ
(*eface)(r).data = f.arg
if len(ityp.mhdr) != 0 {
// convert to interface with methods
// this conversion is guaranteed to succeed - we checked in SetFinalizer
(*iface)(r).tab = assertE2I(ityp, (*eface)(r)._type)
}
default:
throw("bad kind in runfinq")
}
// 调用finalizer函数
fingRunning = true
reflectcall(nil, unsafe.Pointer(f.fn), frame, uint32(framesz), uint32(framesz), uint32(framesz), ®s)
fingRunning = false
// Drop finalizer queue heap references
// before hiding them from markroot.
// This also ensures these will be
// clear if we reuse the finalizer.
// 清空finalizer的属性
f.fn = nil
f.arg = nil
f.ot = nil
atomic.Store(&fb.cnt, i-1)
}
// 将已经完成的finalizer放入finc以作缓存,避免再次分配内存
next := fb.next
lock(&finlock)
fb.next = finc
finc = fb
unlock(&finlock)
fb = next
}
}
}
|
finq中插入finalizer
sweep
在sweep 中有下面一段函数
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
func (s *mspan) sweep(preserve bool) bool {
....
specialp := &s.specials
special :=*specialp
for special != nil {
....
if special.kind == _KindSpecialFinalizer || !hasFin {
// Splice out special record.
y := special
special = special.next
*specialp = special
// 加入全局finq队列的入口就在这里了
freespecial(y, unsafe.Pointer(p), size)
}
....
}
....
}
|
freespecial
在gc的时候,不仅要把special对应的内存释放掉,而且把specials整理创建对应dinalizer对象,并插入到 finq队列里面
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
// freeSpecial performs any cleanup on special s and deallocates it.
// s must already be unlinked from the specials list.
func freeSpecial(s *special, p unsafe.Pointer, size uintptr) {
switch s.kind {
case _KindSpecialFinalizer:
// 把这个finalizer加入到全局队列
sf := (*specialfinalizer)(unsafe.Pointer(s))
queuefinalizer(p, sf.fn, sf.nret, sf.fint, sf.ot)
lock(&mheap_.speciallock)
mheap_.specialfinalizeralloc.free(unsafe.Pointer(sf))
unlock(&mheap_.speciallock)
// 下面两种情况不在分析范围内,省略
case _KindSpecialProfile:
sp := (*specialprofile)(unsafe.Pointer(s))
mProf_Free(sp.b, size)
lock(&mheap_.speciallock)
mheap_.specialprofilealloc.free(unsafe.Pointer(sp))
unlock(&mheap_.speciallock)
case _KindSpecialReachable:
sp := (*specialReachable)(unsafe.Pointer(s))
sp.done = true
// The creator frees these.
default:
throw("bad special kind")
panic("not reached")
}
}
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
|
func queuefinalizer(p unsafe.Pointer, fn *funcval, nret uintptr, fint *_type, ot *ptrtype) {
if gcphase != _GCoff {
// Currently we assume that the finalizer queue won't
// grow during marking so we don't have to rescan it
// during mark termination. If we ever need to lift
// this assumption, we can do it by adding the
// necessary barriers to queuefinalizer (which it may
// have automatically).
throw("queuefinalizer during GC")
}
lock(&finlock)
// 如果finq为空或finq的内部数组已经满了,则从finc或重新分配 来获取block并插入到finq的链表头
if finq == nil || finq.cnt == uint32(len(finq.fin)) {
if finc == nil {
finc = (*finblock)(persistentalloc(_FinBlockSize, 0, &memstats.gcMiscSys))
finc.alllink = allfin
allfin = finc
if finptrmask[0] == 0 {
// Build pointer mask for Finalizer array in block.
// Check assumptions made in finalizer1 array above.
if (unsafe.Sizeof(finalizer{}) != 5*sys.PtrSize ||
unsafe.Offsetof(finalizer{}.fn) != 0 ||
unsafe.Offsetof(finalizer{}.arg) != sys.PtrSize ||
unsafe.Offsetof(finalizer{}.nret) != 2*sys.PtrSize ||
unsafe.Offsetof(finalizer{}.fint) != 3*sys.PtrSize ||
unsafe.Offsetof(finalizer{}.ot) != 4*sys.PtrSize) {
throw("finalizer out of sync")
}
for i := range finptrmask {
finptrmask[i] = finalizer1[i%len(finalizer1)]
}
}
}
// 从finc中移除并获取链表头
block := finc
finc = block.next
// 将从finc获取到的链表挂载到finq的队列头,finq指向新的block
block.next = finq
finq = block
}
// 根据finq.cnt获取索引对应的block
f := &finq.fin[finq.cnt]
atomic.Xadd(&finq.cnt, +1) // Sync with markroots
// 设置相关属性
f.fn = fn
f.nret = nret
f.fint = fint
f.ot = ot
f.arg = p
// 设置唤醒标志
fingwake = true
unlock(&finlock)
}
|
唤醒fing
findrunnable
在 findrunnable 中有一段代码如下:
1
2
3
4
5
6
7
8
9
|
func findrunnable() (gp *g, inheritTime bool) {
// 通过状态位判断是否需要唤醒 fing, 通过wakefing来判断并返回fing
if fingwait && fingwake {
if gp := wakefing(); gp != nil {
// 唤醒g,并从休眠出继续执行
ready(gp, 0, true)
}
}
}
|
wakefing
这里不仅会对状态位 fingwait fingwake做二次判断,而且,如果状态位符合唤醒要求的话,需要重置两个状态位
1
2
3
4
5
6
7
8
9
10
11
|
func wakefing() *g {
var res*g
lock(&finlock)
if fingwait && fingwake {
fingwait = false
fingwake = false
res = fing
}
unlock(&finlock)
return res
}
|
标准库应用
内部上,Go 在 net 以及 net/http 包中确保文件先前的打开与关闭准确无误,并且在 os 包中确保之前创建的进程被正常地释放。这里有一个来自 os 包的例子:
1
2
3
4
5
|
func newProcess(pid int, handle uintptr) *Process {
p := &Process{Pid: pid, handle: handle}
runtime.SetFinalizer(p, (*Process).Release)
return p
}
|
当这个进程被释放,finalizer 也会被移除。
1
2
3
4
5
6
7
|
func (p *Process) release() error {
// NOOP for unix.
p.Pid = -1
// no need for a finalizer anymore
runtime.SetFinalizer(p, nil)
return nil
}
|
Go 同样也在测试中使用 finalizers 确保在垃圾回收器中期望的动作被执行,举个例子,sync 包使用了 finalizers 测试在垃圾回收循环中 pool 是否被清空。
业务场景
在日常项目的开发过程中, 总会使用后台goroutine做一些定期清理或者更新的任务, 这就涉及到goroutine生命周期的管理。
处理方式
对于和主程序生命周期基本一致的后台goroutine,一般采用如下显式的Stop()
来进行优雅退出:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
|
type IApp interface {
//...
Stop()
}
type App struct {
// some vars
running bool
stop chan struct{}
onStopped func()
}
func New() *App {
app := &App{
running: true,
stop: make(chan struct{}),
}
go watch()
return app
}
func (app *App) watch() {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-app.stop:
if app.onStopped != nil {
app.onStopped()
}
return
case <-ticker.C:
//do something
}
}
}
func (app *App) Stop() {
if !app.running {
return
}
close(app.stop)
}
|
这种方式除了需要在程序终止之前显式调用一下Stop()
, 没有其他的问题。但是在其他的一些场景中, 你可能就会confuse了
比如我现在想实现一个cache模块,功能和接口都很简单:
1
2
3
4
|
type Cache interface {
Get(key string) (interface{}, bool)
Set(key string, value interface{})
}
|
由于需要定时清理过期的缓存, 所以会使用一个后台goroutine来执行清理的工作, 但是这些应该是对使用者透明的, 不过往往总会出现一些意料之外的结果:
1
2
3
4
5
6
7
8
|
func main() {
c := cache.New()
c.Set("key1", obj)
val, exist := c.Get("key1")
// ...
c = nil
// do other things
}
|
在使用者看来, cache已经没有引用了, 会在gc的时候被回收。 但实际上由于后台goroutine的存在, cache始终不能满足不可达的条件, 也就不会被gc回收, 从而产生了内存泄露的问题。
解决这个问题当前也可以按照上面的方式, 显式增加一个Close()
方法, 靠channel通知关闭goroutine, 但是这无疑增加了使用成本, 而且也不能避免使用者忘记Close()
这种场景。
还有没有更好的方式,不需要用户显式关闭, 在检查到没有引用之后, 主动终止goroutine,等待gc回收? 当然。 runtime.SetFinalizer
可以帮助我们达到这个目的。如何利用SetFinalizer来进行cache后台goroutine的清理呢?
istio的中lrucache给了我们一种巧妙的方式:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
type lruWrapper struct {
*lruCache
}
// We return a 'see-through' wrapper for the real object such that
// the finalizer can trigger on the wrapper. We can't set a finalizer
// on the main cache object because it would never fire, since the
// evicter goroutine is keeping it alive
result := &lruWrapper{c}
runtime.SetFinalizer(result, func(w *lruWrapper) {
w.stopEvicter <- true
w.evicterTerminated.Wait()
})
|
在lrucache外面加上一层wrapper, lrucache作为wrapper的匿名字段存在, 并且在wrapper上注册了SetFinalizer函数来终止后台的goroutine。 由于后台goroutine是和lrucache关联的, 当没有引用指向wrapper的时候, gc就会执行关联的SetFinalizer终止lrucache的后台goroutine,这样最终lrucache也会变成不可达的状态, 被gc回收。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
|
type Cache = *wrapper
type wrapper struct {
*cache
}
type cache struct {
content string
stop chan struct{}
onStopped func()
}
func newCache() *cache {
return &cache{
content: "some thing",
stop: make(chan struct{}),
}
}
func NewCache() Cache {
w := &wrapper{
cache : newCache(),
}
go w.cache.run()
runtime.SetFinalizer(w, (*wrapper).stop)
return w
}
func (w *wrapper) stop() {
w.cache.stop()
}
func (c *cache) run() {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
// do some thing
case <-c.stop:
if c.onStopped != nil {
c.onStopped()
}
return
}
}
}
func (c *cache) stop() {
close(c.stop)
}
|
对于对象是否被回收, 最靠谱的方式就是靠test来检测并保证这一行为:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
func TestFinalizer(t *testing.T) {
s := assert.New(t)
w := NewCache()
var cnt int = 0
stopped := make(chan struct{})
w.onStopped = func() {
cnt++
close(stopped)
}
s.Equal(0, cnt)
w = nil
runtime.GC()
select {
case <-stopped:
case <-time.After(10 * time.Second):
t.Fail()
}
s.Equal(1, cnt)
}
|
弊端
有些同学喜欢利用 runtime.SetFinalizer 模拟析构函数,当变量被回收时,执行一些回收操作,加速一些资源的释放。在做性能优化的时候这样做确实有一定的效果,不过这样做是有一定的风险的。
比如下面这段代码,初始化一个文件描述符,当 GC 发生时释放掉无效的文件描述符。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
package main
import (
"runtime"
"syscall"
)
type File struct{ d int }
func main() {
p := openFile("t.txt")
content := readFile(p.d)
println("Here is the content: " + content)
}
func openFile(path string) *File {
d, err := syscall.Open(path, syscall.O_RDONLY, 0)
if err != nil {
panic(err)
}
p := &File{d}
runtime.SetFinalizer(p, func(p *File) {
syscall.Close(p.d)
})
return p
}
func readFile(descriptor int) string {
doSomeAllocation()
var buf [1000]byte
_, err := syscall.Read(descriptor, buf[:])
if err != nil {
panic(err)
}
return string(buf[:])
}
func doSomeAllocation() {
var a *int
// memory increase to force the GC
for i := 0; i < 10000000; i++ {
i := 1
a = &i
}
_ = a
}
|
这个程序中一个函数打开文件,另一个函数读取文件。代表文件的结构体注册了一个 finalizer,在 gc 释放结构体时自动关闭文件。运行这个程序,会出现 panic:
1
2
3
4
5
6
7
8
|
panic: bad file descriptor
goroutine 1 [running]:
main.readFile(0x3, 0x5, 0xc000078008)
main.go:42 +0x103
main.main()
main.go:14 +0x4b
exit status 2
|
下面是流程图:
- 打开文件,返回一个文件描述符
- 这个文件描述符被传递给读取文件的函数
- 这个函数首先做一些繁重的工作:

allocate 函数触发 gc:

因为文件描述符是个整型,并以副本传递,所以打开文件的函数返回的结构体 *File*
不再被引用。Gc 把它标记为可以被回收的。之后触发这个变量注册的 finalizer,关闭文件。
然后,主协程读取文件:

因为文件已经被 finalizer 关闭,所以会出现 panic。
runtime.KeepAlive
runtime 包暴露了一个方法,用来在 Go 程序中避免出现这种情况,并显式地声明了让变量不被回收。在运行到这个调用这个方法的地方之前,gc 不会清除指定的变量。下面是加了对这个方法的调用的新代码:
1
2
3
4
5
6
7
8
|
func main() {
p := openFile("t.txt")
content := readFile(p.d)
runtime.KeepAlive(p)
println("Here is the content: "+content)
}
|
runtime.KeepAlive 能阻止 runtime.SetFinalizer 延迟发生,保证我们的变量不被 GC 所回收。
keepAlive 方法本身没有做什么:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
|
// Mark KeepAlive as noinline so that it is easily detectable as an intrinsic.
//go:noinline
// KeepAlive marks its argument as currently reachable.
// This ensures that the object is not freed, and its finalizer is not run,
// before the point in the program where KeepAlive is called.
//
// A very simplified example showing where KeepAlive is required:
// type File struct { d int }
// d, err := syscall.Open("/file/path", syscall.O_RDONLY, 0)
// // ... do something if err != nil ...
// p := &File{d}
// runtime.SetFinalizer(p, func(p *File) { syscall.Close(p.d) })
// var buf [10]byte
// n, err := syscall.Read(p.d, buf[:])
// // Ensure p is not finalized until Read returns.
// runtime.KeepAlive(p)
// // No more uses of p after this point.
//
// Without the KeepAlive call, the finalizer could run at the start of
// syscall.Read, closing the file descriptor before syscall.Read makes
// the actual system call.
//
// Note: KeepAlive should only be used to prevent finalizers from
// running prematurely. In particular, when used with unsafe.Pointer,
// the rules for valid uses of unsafe.Pointer still apply.
func KeepAlive(x interface{}) {
// Introduce a use of x that the compiler can't eliminate.
// This makes sure x is alive on entry. We need x to be alive
// on entry for "defer runtime.KeepAlive(x)"; see issue 21402.
if cgoAlwaysFalse {
println(x)
}
}
|
运行时,Go 编译器会以很多种方式优化代码:函数内联,死码消除,等等。这个函数不会被内联,Go 编译器可以轻易地探测到哪里调用了 keepAlive。编译器很容易追踪到调用它的地方,它会发出一个特殊的 SSA 指令,以此来确保它不会被 gc 回收。
在生成的 SSA 代码中也可以看到这个 SSA 指令:

实际上,它不是作为一个被调用的函数,而是由 ssa.go 实现的编译器内部实现,类似于 unsafe.Pointer
。当你的代码中使用了 runtime.KeepAlive()
,Go 编译器会设置一个名为 OpKeepAlive
的静态单赋值(SSA),然后剩余的编译就会知道将这个变量的存活期保证到使用了 runtime.KeepAlive()
的时刻。
runtime.KeepAlive() 是一个特别的魔法有一个直接的后果就是你不能得到它的地址。如果你这样做的话, Go 会报错:
1
|
./tst.go:20:22: cannot take the address of runtime.KeepAlive
|
我不知道 Go 是否会聪明地优化掉一个只调用 runtime.KeepAlive 的函数, 但希望你永远不需要间接调用 runtime.KeepAlive
。
参考
深入理解Go-runtime.SetFinalizer原理剖析
Go如何巧妙使用runtime.SetFinalizer
使用runtime.SetFinalizer优雅关闭后台goroutine
Go: Finalizers 怎么使用?
Go: 延长变量的生命周期