1.回顾切片

上一篇文章我们从源码的角度分析了切片,包括切片的数据结构,底层实现,扩容以及添加等,但是我们并没有详细分析切片扩容的规则到底是什么?尽管上一篇文章中展示了一些代码,可是为什么扩容结果是这样呢?今天我们就来详细的分析一波。

1.1. 示例代码

再回顾一下上一篇文章中关于扩容的代码:

s := []string{"a", "b"} // 此时切片长度为2,容量也为2。
s = append(s, "c")
s = append(s, "d")
s = append(s, "e")
fmt.Printf("len=%d, cap=%d", len(s), cap(s)) // 结果:len=5, cap=8

s := []string{"a", "b"} // 此时切片长度为2,容量也为2。
s = append(s, "c", "d", "e")
fmt.Printf("len=%d, cap=%d", len(s), cap(s)) // 结果:len=5, cap=5

s := []int{1, 2} // 此时切片长度为2,容量也为2。
s = append(s, 3, 4, 5)
fmt.Printf("len=%d, cap=%d", len(s), cap(s)) // 结果:len=5, cap=6

2.扩容源码

2.1. 源码

源码位置在go/src/runtime/slice.go

func growslice(et *_type, old slice, cap int) slice {
if raceenabled {
    callerpc := getcallerpc()
    racereadrangepc(old.array, uintptr(old.len*int(et.size)), callerpc, funcPC(growslice))
  }
if msanenabled {
    msanread(old.array, uintptr(old.len*int(et.size)))
  }

if cap < old.cap {
panic(errorString("growslice: cap out of range"))
  }

if et.size == 0 {
// append should not create a slice with nil pointer but non-zero len.
// We assume that append doesn't need to preserve old.array in this case.
return slice{unsafe.Pointer(&zerobase), old.len, cap}
  }

  newcap := old.cap
  doublecap := newcap + newcap
if cap > doublecap {
    newcap = cap
  } else {
if old.len < 1024 {
      newcap = doublecap
    } else {
// Check 0 < newcap to detect overflow
// and prevent an infinite loop.
for 0 < newcap && newcap < cap {
        newcap += newcap / 4
      }
// Set newcap to the requested cap when
// the newcap calculation overflowed.
if newcap <= 0 {
        newcap = cap
      }
    }
  }

var overflow bool
var lenmem, newlenmem, capmem uintptr
// Specialize for common values of et.size.
// For 1 we don't need any division/multiplication.
// For sys.PtrSize, compiler will optimize division/multiplication into a shift by a constant.
// For powers of 2, use a variable shift.
switch {
case et.size == 1:
    lenmem = uintptr(old.len)
    newlenmem = uintptr(cap)
    capmem = roundupsize(uintptr(newcap))
    overflow = uintptr(newcap) > maxAlloc
    newcap = int(capmem)
case et.size == sys.PtrSize:
    lenmem = uintptr(old.len) * sys.PtrSize
    newlenmem = uintptr(cap) * sys.PtrSize
    capmem = roundupsize(uintptr(newcap) * sys.PtrSize)
    overflow = uintptr(newcap) > maxAlloc/sys.PtrSize
    newcap = int(capmem / sys.PtrSize)
case isPowerOfTwo(et.size):
var shift uintptr
if sys.PtrSize == 8 {
// Mask shift for better code generation.
      shift = uintptr(sys.Ctz64(uint64(et.size))) & 63
    } else {
      shift = uintptr(sys.Ctz32(uint32(et.size))) & 31
    }
    lenmem = uintptr(old.len) << shift
    newlenmem = uintptr(cap) << shift
    capmem = roundupsize(uintptr(newcap) << shift)
    overflow = uintptr(newcap) > (maxAlloc >> shift)
    newcap = int(capmem >> shift)
default:
    lenmem = uintptr(old.len) * et.size
    newlenmem = uintptr(cap) * et.size
    capmem, overflow = math.MulUintptr(et.size, uintptr(newcap))
    capmem = roundupsize(capmem)
    newcap = int(capmem / et.size)
  }

// The check of overflow in addition to capmem > maxAlloc is needed
// to prevent an overflow which can be used to trigger a segfault
// on 32bit architectures with this example program:
//
// type T [1<<27 + 1]int64
//
// var d T
// var s []T
//
// func main() {
//   s = append(s, d, d, d, d)
//   print(len(s), "\n")
// }
if overflow || capmem > maxAlloc {
panic(errorString("growslice: cap out of range"))
  }

var p unsafe.Pointer
if et.ptrdata == 0 {
    p = mallocgc(capmem, nil, false)
// The append() that calls growslice is going to overwrite from old.len to cap (which will be the new length).
// Only clear the part that will not be overwritten.
    memclrNoHeapPointers(add(p, newlenmem), capmem-newlenmem)
  } else {
// Note: can't use rawmem (which avoids zeroing of memory), because then GC can scan uninitialized memory.
    p = mallocgc(capmem, et, true)
if lenmem > 0 && writeBarrier.enabled {
// Only shade the pointers in old.array since we know the destination slice p
// only contains nil pointers because it has been cleared during alloc.
      bulkBarrierPreWriteSrcOnly(uintptr(p), uintptr(old.array), lenmem)
    }
  }
  memmove(p, old.array, lenmem)

return slice{p, old.len, newcap}
}

这里我把所有代码都拿过来了,在详细分析这部分代码时,我们先准备一下必备的基础知识。

2.2. 基础知识

2.2.1. 分配内存

go 语言的内存分配也是比较复杂的,这篇文章不会把所有内存分配的细节讲出来,只会把和切片扩容相关的知识点说一下,首先我们先看一个源码go/src/runtime/sizeclasses.go

// Code generated by mksizeclasses.go; DO NOT EDIT.
//go:generate go run mksizeclasses.go

package runtime

// class  bytes/obj  bytes/span  objects  tail waste  max waste
//     1          8        8192     1024           0     87.50%
//     2         16        8192      512           0     43.75%
//     3         32        8192      256           0     46.88%
//     4         48        8192      170          32     31.52%
//     5         64        8192      128           0     23.44%
//     6         80        8192      102          32     19.07%
//     7         96        8192       85          32     15.95%
//     8        112        8192       73          16     13.56%
//     9        128        8192       64           0     11.72%
//    10        144        8192       56         128     11.82%
//    11        160        8192       51          32      9.73%
//    12        176        8192       46          96      9.59%
//    13        192        8192       42         128      9.25%
//    14        208        8192       39          80      8.12%
//    15        224        8192       36         128      8.15%
//    16        240        8192       34          32      6.62%
//    17        256        8192       32           0      5.86%
//    18        288        8192       28         128     12.16%
//    19        320        8192       25         192     11.80%
//    20        352        8192       23          96      9.88%
//    21        384        8192       21         128      9.51%
//    22        416        8192       19         288     10.71%
//    23        448        8192       18         128      8.37%
//    24        480        8192       17          32      6.82%
//    25        512        8192       16           0      6.05%
//    26        576        8192       14         128     12.33%
//    27        640        8192       12         512     15.48%
//    28        704        8192       11         448     13.93%
//    29        768        8192       10         512     13.94%
//    30        896        8192        9         128     15.52%
//    31       1024        8192        8           0     12.40%
//    32       1152        8192        7         128     12.41%
//    33       1280        8192        6         512     15.55%
//    34       1408       16384       11         896     14.00%
//    35       1536        8192        5         512     14.00%
//    36       1792       16384        9         256     15.57%
//    37       2048        8192        4           0     12.45%
//    38       2304       16384        7         256     12.46%
//    39       2688        8192        3         128     15.59%
//    40       3072       24576        8           0     12.47%
//    41       3200       16384        5         384      6.22%
//    42       3456       24576        7         384      8.83%
//    43       4096        8192        2           0     15.60%
//    44       4864       24576        5         256     16.65%
//    45       5376       16384        3         256     10.92%
//    46       6144       24576        4           0     12.48%
//    47       6528       32768        5         128      6.23%
//    48       6784       40960        6         256      4.36%
//    49       6912       49152        7         768      3.37%
//    50       8192        8192        1           0     15.61%
//    51       9472       57344        6         512     14.28%
//    52       9728       49152        5         512      3.64%
//    53      10240       40960        4           0      4.99%
//    54      10880       32768        3         128      6.24%
//    55      12288       24576        2           0     11.45%
//    56      13568       40960        3         256      9.99%
//    57      14336       57344        4           0      5.35%
//    58      16384       16384        1           0     12.49%
//    59      18432       73728        4           0     11.11%
//    60      19072       57344        3         128      3.57%
//    61      20480       40960        2           0      6.87%
//    62      21760       65536        3         256      6.25%
//    63      24576       24576        1           0     11.45%
//    64      27264       81920        3         128     10.00%
//    65      28672       57344        2           0      4.91%
//    66      32768       32768        1           0     12.50%

const (
  _MaxSmallSize   = 32768
  smallSizeDiv    = 8
  smallSizeMax    = 1024
  largeSizeDiv    = 128
  _NumSizeClasses = 67
  _PageShift      = 13
)

var class_to_size = [_NumSizeClasses]uint16{0, 8, 16, 32, 48, 64, 80, 96, 112, 128, 144, 160, 176, 192, 208, 224, 240, 256, 288, 320, 352, 384, 416, 448, 480, 512, 576, 640, 704, 768, 896, 1024, 1152, 1280, 1408, 1536, 1792, 2048, 2304, 2688, 3072, 3200, 3456, 4096, 4864, 5376, 6144, 6528, 6784, 6912, 8192, 9472, 9728, 10240, 10880, 12288, 13568, 14336, 16384, 18432, 19072, 20480, 21760, 24576, 27264, 28672, 32768}
var class_to_allocnpages = [_NumSizeClasses]uint8{0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 3, 2, 3, 1, 3, 2, 3, 4, 5, 6, 1, 7, 6, 5, 4, 3, 5, 7, 2, 9, 7, 5, 8, 3, 10, 7, 4}

type divMagic struct {
  shift    uint8
  shift2   uint8
  mul      uint16
  baseMask uint16
}

var class_to_divmagic = [_NumSizeClasses]divMagic{{0, 0, 0, 0}, {3, 0, 1, 65528}, {4, 0, 1, 65520}, {5, 0, 1, 65504}, {4, 11, 683, 0}, {6, 0, 1, 65472}, {4, 10, 205, 0}, {5, 9, 171, 0}, {4, 11, 293, 0}, {7, 0, 1, 65408}, {4, 13, 911, 0}, {5, 10, 205, 0}, {4, 12, 373, 0}, {6, 9, 171, 0}, {4, 13, 631, 0}, {5, 11, 293, 0}, {4, 13, 547, 0}, {8, 0, 1, 65280}, {5, 9, 57, 0}, {6, 9, 103, 0}, {5, 12, 373, 0}, {7, 7, 43, 0}, {5, 10, 79, 0}, {6, 10, 147, 0}, {5, 11, 137, 0}, {9, 0, 1, 65024}, {6, 9, 57, 0}, {7, 9, 103, 0}, {6, 11, 187, 0}, {8, 7, 43, 0}, {7, 8, 37, 0}, {10, 0, 1, 64512}, {7, 9, 57, 0}, {8, 6, 13, 0}, {7, 11, 187, 0}, {9, 5, 11, 0}, {8, 8, 37, 0}, {11, 0, 1, 63488}, {8, 9, 57, 0}, {7, 10, 49, 0}, {10, 5, 11, 0}, {7, 10, 41, 0}, {7, 9, 19, 0}, {12, 0, 1, 61440}, {8, 9, 27, 0}, {8, 10, 49, 0}, {11, 5, 11, 0}, {7, 13, 161, 0}, {7, 13, 155, 0}, {8, 9, 19, 0}, {13, 0, 1, 57344}, {8, 12, 111, 0}, {9, 9, 27, 0}, {11, 6, 13, 0}, {7, 14, 193, 0}, {12, 3, 3, 0}, {8, 13, 155, 0}, {11, 8, 37, 0}, {14, 0, 1, 49152}, {11, 8, 29, 0}, {7, 13, 55, 0}, {12, 5, 7, 0}, {8, 14, 193, 0}, {13, 3, 3, 0}, {7, 14, 77, 0}, {12, 7, 19, 0}, {15, 0, 1, 32768}}
var size_to_class8 = [smallSizeMax/smallSizeDiv + 1]uint8{0, 1, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 18, 18, 18, 19, 19, 19, 19, 20, 20, 20, 20, 21, 21, 21, 21, 22, 22, 22, 22, 23, 23, 23, 23, 24, 24, 24, 24, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31}
var size_to_class128 = [(_MaxSmallSize-smallSizeMax)/largeSizeDiv + 1]uint8{31, 32, 33, 34, 35, 36, 36, 37, 37, 38, 38, 39, 39, 39, 40, 40, 40, 41, 42, 42, 43, 43, 43, 43, 43, 44, 44, 44, 44, 44, 44, 45, 45, 45, 45, 46, 46, 46, 46, 46, 46, 47, 47, 47, 48, 48, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 53, 53, 53, 53, 54, 54, 54, 54, 54, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 58, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 59, 60, 60, 60, 60, 60, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 61, 62, 62, 62, 62, 62, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66, 66}

2.2.1.1 介绍内存对齐

首先我们需要知道 CPU 从内存度数据是按照每次固定大小读的,例如每次读8字节或者4字节,所以就有一个效率问题,每读取一次,就浪费一点时间,因此 Go 中也存在内存对齐,今天重点不是内存对齐,但是这里可以先和大家简单说一下:每个对象在内存中都要占一块空间,可能是1字节,可能是3字节,可能是8字节,所以对于一个struct 来说,其字段类型会觉得这个结构体对象所占的内存,但是所占内存大小可能不是所有字段类型所占内存的大小,因为为了让 CPU 读取方便,我们就会对每个字段进行内存对齐。点到为止,上面的内存对齐就不深说了,但是上面所说的内容大家还是需要理解的。

2.2.1.2. 分配内存事实

当我们创建一个对象时,需要分配一块内存。假设我们创建的对象需要52 byte,系统是不会真的就给我们52byte大小的内存的,首先会根据上面代码中第六行注释的部分来计算需要给你分配的内存大小,这里可以参考一下go/src/runtime/msize.go 中的roundupsize 函数。也就是需要向上取整,意思就是48<52<64,所以会分配64byte大小的内存。

这里我们需要明白一个事实,如果我们每次创建一个对象,程序都向计算机中申请一块内存,这样在程序运行时我们是会频繁的创建对象的,这样效率会大大降低,所以程序会预先申请好一些内存块,其大小就是:8、16、32、48等等,这样在我们让程序申请内存时,程序就可以把申请好的内存选一块给我们了,效率也就提高了。

3.扩容规则

3.1.分析

我们使用下面的代码来分析扩容的规则:

s := []int64{1, 2}
s = append(s, 3, 4, 5)
fmt.Printf("len=%d, cap=%d", len(s), cap(s)) // len=5, cap=6

我们首先知道int64 类型大小为8字节。我们再看切片源码的参数:func growslice(et *_type, old slice, cap int) slice,在上面的代码中,本地扩容的参数第一个是int64类型,第二个就是扩容前切片a(元素为1和2),第三个参数就是预估容量5(因为原有切片容量加上新加元素个数就是5),我们再继续看源码中对容量计算的部分代码:

...
newcap := old.cap
doublecap := newcap + newcap
if cap > doublecap {
  newcap = cap
} else {
if old.len < 1024 {
    newcap = doublecap
  } else {
// Check 0 < newcap to detect overflow
// and prevent an infinite loop.
for 0 < newcap && newcap < cap {
      newcap += newcap / 4
    }
// Set newcap to the requested cap when
// the newcap calculation overflowed.
if newcap <= 0 {
      newcap = cap
    }
  }
}
...

newcap 开始是就切片的容量2,doublecap 为2+2=4,cap 为5,此时cap>doublecap,因此newcap 最终就是5(剩下的逻辑自己可以看看,如果 cap<doublecap 的情况自己可以试一下)。

在 2.2.1.2 中我们介绍了 roundupsize,此时我们已经知道 newcap 为5,int64 占用8字节,那就是说我切片扩容后需要5*8=40字节,再通过 roundupsize 计算,最终给我们的内存大小应该是48字节(可以回顾一下2.2.1.2),48/8=6,所以结果我们就知道了,最终切片的 cap 就是6。

源码实现的方式中很多计算,包括左移右移等,但是想法和我上面说的是一样的,感兴趣大家可以参考源码计算一下。

4.总结

至此切片扩容就结束了,其中有一些地方这里只是简单说了下。根据上面的例子,大家可以手动试一下string 类型,或者int32等。如果对你有帮助点个在看哦~