从零开始-Golang

文章开辟于《Golang 区块链入门到实战_以太坊/fabric》[1]

没成想半途真的投入 Golang 岗, 深挖亿下…

分割线

配置

常用命令

# windows 安装
scoop install go-cn

# 模块初始化
go mod init ProjectName

# 依赖更新
go get -u xxx
# Go 1.16 及以后版本
go install xxx
go mod tidy

# 显示已安装的package
go list ...
gopkgs

cannot determine module path for source directory (outside GOPATH[3]


Go-env

可以通过设置系统环境变量和使用 go env -w 两种形式 [2], 前者权限大于后者, 推荐用后者 (已经测试好的配置, 尽量别改了)

# 代理服务列表
[Environment]::SetEnvironmentVariable('GOPROXY', 'https://proxy.golang.com.cn,direct', 'User')
# GOSUMDB 建议留空取默认的sum.golang.org, 速度慢点总比疯狂报错好
[Environment]::SetEnvironmentVariable('GOSUMDB', '', 'User')

# # Go 1.15 及之前版本
go env -w GO111MODULE=on
# 下面二选一
go env -w GOPROXY=https://proxy.golang.com.cn,direct
go env -w GOPROXY=https://goproxy.cn,direct
go env -w GOSUMDB=

# 已失效
go env -w GOPROXY=https://goproxy.io,direct
  • 通过 go env 查看更改后的 (有时可能要重启才生效)

    set GO111MODULE=on
    set GOARCH=amd64
    set GOBIN=
    set GOCACHE=C:\Users\Administrator\AppData\Local\go-build
    set GOENV=C:\Users\Administrator\AppData\Roaming\go\env
    set GOEXE=.exe
    set GOEXPERIMENT=
    set GOFLAGS=
    set GOHOSTARCH=amd64
    set GOHOSTOS=windows
    set GOINSECURE=
    set GOMODCACHE=D:\Scoop\apps\go-cn\current\global_path\pkg\mod
    set GONOPROXY=codeup.aliyun.com
    set GONOSUMDB=codeup.aliyun.com
    set GOOS=windows
    set GOPATH=D:\Scoop\apps\go-cn\current\global_path
    set GOPRIVATE=codeup.aliyun.com
    set GOPROXY=https://proxy.golang.com.cn,direct
    set GOROOT=D:\Scoop\apps\go-cn\current
    set GOSUMDB=sum.golang.org
    set GOTMPDIR=
    set GOTOOLDIR=D:\Scoop\apps\go-cn\current\pkg\tool\windows_amd64
    set GOVCS=
    set GOVERSION=go1.19
    set GCCGO=gccgo
    set GOAMD64=v1
    set AR=ar
    set CC=gcc
    set CXX=g++
    set CGO_ENABLED=1
    set GOMOD=NUL
    set GOWORK=
    set CGO_CFLAGS=-g -O2
    set CGO_CPPFLAGS=
    set CGO_CXXFLAGS=-g -O2
    set CGO_FFLAGS=-g -O2
    set CGO_LDFLAGS=-g -O2
    set PKG_CONFIG=pkg-config
    set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\ADMINI~1\AppData\Local\Temp\go-build2787993248=/tmp/go-build -gno-record-gcc-switches

Goland-快捷键迁移

在另一篇: 🤔Matters-found-in-IDEs


项目热部署

这里推荐用 air, 项目还比较活跃

  1. 先安装上

  2. 进入项目 air init, 会出现 .air.toml

  3. 配置 .air.toml, 实际上只需要改一些 bin 和 cmd, 如下

    root = "."
    testdata_dir = ""
    tmp_dir = ""

    [build]
    args_bin = []
    bin = "discord-bot.exe"
    cmd = "go build -o . codeup.aliyun.com/wenuts/aimage/discord-bot"
    delay = 1000
    exclude_dir = ["assets", "tmp", "vendor", "testdata"]
    exclude_file = []
    exclude_regex = ["_test.go"]
    exclude_unchanged = false
    follow_symlink = false
    full_bin = ""
    include_dir = []
    include_ext = ["go", "tpl", "tmpl", "html"]
    kill_delay = "0s"
    log = "build-errors.log"
    send_interrupt = false
    stop_on_error = true

    [color]
    app = ""
    build = "yellow"
    main = "magenta"
    runner = "green"
    watcher = "cyan"

    [log]
    time = false

    [misc]
    clean_on_exit = false

    [screen]
    clear_on_rebuild = false
  4. console 直接运行 air, 就可以热部署开发了, 缺点是不能 Debug


go-install

给出地址, go 可以直接拉下来编译为对应平台的可执行文件

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install github.com/cuigh/protoc-gen-auxo@latest
go install github.com/tpng/gopkgs@latest

分割线

learning

基础语法可以跟着菜鸟教程学习 [5]

package

package-demo

  • package被误改会报错[6]

    package command-line-arguments is not a main package

    正确 import 法:[7]


  • 简易例子:

    • test.go

      package main

      func sum( a, b int ) int {
      return a + b
      }
    • main.go

      package main

      func main() {
      fmt.Println(sum(1, 2))
      }

同一包下同名方法隔离

比如 a.go 和 b.go 都在 package main 里, 都有 print()方法, 没隔离的情况下会报错 (同名方法重复)

  • 一般常采用接口进行文件间的隔离:

    type A struct {
    }

    func (a *A) print() {}
    type B struct {
    }

    func (b *B) print() {}

副作用导入

有时项目会用到其他 package 的初始化, 但并没有实际调用函数, 就会通过 _ 副作用导入, 不写不会提示但缺少运行时会报错, 除非熟悉不然很难揪出来的问题

import (
_ "codeup.aliyun.com/wenuts/aimage/api/handler"
_ "codeup.aliyun.com/wenuts/aimage/api/security"
"github.com/cuigh/auxo/app"
"github.com/cuigh/auxo/app/flag"
"github.com/cuigh/auxo/app/ioc"
_ "github.com/cuigh/auxo/cache/memory"
_ "github.com/cuigh/auxo/cache/redis"
"github.com/cuigh/auxo/config"
"github.com/cuigh/auxo/data/valid"
"github.com/cuigh/auxo/errors"
_ "github.com/cuigh/auxo/net/rpc/codec/proto"
_ "github.com/cuigh/auxo/net/rpc/resolver/dns"
"github.com/cuigh/auxo/net/web"
"github.com/cuigh/auxo/net/web/filter"
"net/http"
"os"
)

public-package

发布到网上的库可以通过 go 获取: go install github.com/Weidows/Golang/demo/gmm@latest, 有的会遇到报错:

 go install github.com/Weidows/Golang/demo/gmm@latest                                                                                                                                                                   go: github.com/Weidows/Golang/demo/gmm@latest: github.com/Weidows/Golang@v0.0.0-20221031074205-850af8522b95: parsing go.mod:
module declares its path as: Golang
but was required as: github.com/Weidows/Golang

原因是 go.mod 里面第一行应换为 module github.com/Weidows/Golang 形式


http-json-req

用 golang 做带有 JSON 格式 body 的 http 请求

这个问题同时出现的

func main() {
token := "AOEOulb6yc81T2dbV8QN2-r9-RS7BoEEGHJhHkfLp50kMB8F0fO61PzHkQUiT1qMTMQZwAqHIUlU4D89gUs_YwX1BVc5KbD1maDC2M_T0zyk8BKSKGBrsYf5bqx606Xc3vSu3qYRv-jDVcb7D52Kqpl3M-jWTeaDhcSm2CyolUMie3oyifyh3fE"

jsonBody, _ := json.Marshal(map[string]any{
"grantType": "refresh_token",
"refreshToken": token,
})
req := bytes.NewBuffer(jsonBody)

//resp, _ := http.Post("", "", buff)
resp := req

var m any
_ = json.NewDecoder(resp).Decode(&m)
// map[grantType:refresh_token refreshToken:AOEOulb6yc81T2dbV8QN2-r9-RS7BoEEGHJhHkfLp50kMB8F0fO61PzHkQUiT1qMTMQZwAqHIUlU4D89gUs_YwX1BVc5KbD1maDC2M_T0zyk8BKSKGBrsYf5bqx606Xc3vSu3qYRv-jDVcb7D52Kqpl3M-jWTeaDhcSm2CyolUMie3oyifyh3fE]
fmt.Println(m)
}

名称规范

  1. 源文件名不包含大写

    get.go -> × getV3.go
    get.go -> √ get3.go

  2. 驼峰命名, 对于公开方法/变量/常量, 首字母大写,否则小写


指针

go 语言指针符号的*和&

  • golang string 转 *string

    比如字符串 “abc”, 无法直接通过 &“abc” 取到 *string, 需要先出个变量

    v := “abc”, &v 就可以取到了

分割线

gp-python

# github.com/sbinet/go-python
In file included from D:\Scoop\apps\go-cn\current\global_path\pkg\mod\github.com\sbinet\go-python@v0.1.0\capi.go:3:
./go-python.h:4:10: fatal error: Python.h: No such file or directory
4 | #include "Python.h"
| ^~~~~~~~~~
compilation terminated.

因为 go-python 调的是 python2, 不兼容 python3, 可以用 cpy

pkg-config: exec: "pkg-config": executable file not found in %PATH%

没装这个软件

分割线

concurrent

并发模型实体通信方式优点缺点
多进程进程socket, 共享内存, 管道, 信号量, unix 域等隔离性好, 因为每个进程都有自己独立的进程空间统一性差, 即数据同步比较麻烦. 解决方案(消息队列 zeromq 解决最终一致性问题, rpc 解决强一致性问题, zookeeper 解决服务协调的问题)
多线程线程消息队列, 管道, 锁等统一性强, 因为线程都在同一个进程内(这里的多线程是指同一进程内的多线程)隔离性差, 线程间共享了很多资源, 并且可以轻易的访问其他线程的私有空间, 需要使用锁来进行控制(锁的类型选择和粒度控制都是比较难的)
csp协程channel (可理解为加强版多线程解决方案)
actoractor通过消息队列传递指针即可达到通信目的
  • Golang 对 CSP(Communicating Sequential Process) 模型借用了 process(goroutine) 和 channel 这两个概念来实现自己的并发模型, 是对线程的抽象

    如下一个简易 csp golang 实现

    var ch = make(chan string)

    func receive() {
    // 异步, 阻塞读
    msg := <-ch
    fmt.Println(msg)
    }

    func send() {
    // 写
    ch <- "Hello,CSP."
    }

    func main() {
    go send()
    go receive()
    }
  • actor 是从语言层面抽象出来的进程概念, erlang 是从语言层面来实现 actor 模型(可理解为加强版多进程解决方案), actor 有三个组成:

    隔离环境: 内存块或 lua 虚拟机

    回调函数: 用于执行 actor, 消费消息

    消息队列: 用于存储消息


context

context 上下文不复用, 多数是用来设置超时调用的, 调用一次方法/每个调用链生成一次, 不然后面调用的方法可能会因为时间短而超时失败


cgo

  • 依赖引入比较棘手, 下面分别为代码与环境变量需要配置的

    /*
    #cgo CFLAGS: -I../face_swap/include
    #cgo LDFLAGS: -L/root/workspace/src/face_swap/lib -lSwapFace
    #include "../face_swap/include/sim_swap_api.h"
    */
    export LD_LIBRARY_PATH=/root/workspace/src/face_swap/lib:/usr/local/cuda/lib64:/usr/local/OpenCV/lib:/usr/local/TensorRT/lib:/usr/local/LibTorch/lib:/usr/local/3rdlib
  • CFLAGS (compile flags):

    头文件只有编译时需要, 可以相对路径引入

  • LDFLAGS (load flags):

    这项由于 gcc 设计问题, 必须使用绝对路径引用, 而且路径会编译进二进制文件, 在运行环境也会去找对应文件 [10]

    比如上面在运行时就会去找 /root/workspace/src/face_swap/lib/libSwapFace.so

  • LD_LIBRARY_PATH (load library path):

    与 LDFLAGS 不同, 这个搜索的是链接库路径, LDFLAGS 是指定的链接文件, 所以写在代码 LDFLAGS 那里无效

    无论是代码中还是 libSwapFace.so 的依赖项在查找时都是从 LD_LIBRARY_PATH 这些路径里面找


zincsearch

  • 虽然像是 mongo 不指定字段也能直接往里塞数据, 但是只能在创建 index 时对字段的类型和策略进行指定: Update Mapping - ZincSearch, 如下:

    {
    "properties": {
    "@timestamp": {
    "type": "date",
    "index": true,
    "store": false,
    "sortable": true,
    "aggregatable": true,
    "highlightable": false
    },
    "_id": {
    "type": "keyword",
    "index": true,
    "store": false,
    "sortable": true,
    "aggregatable": true,
    "highlightable": false
    },
    "id": {
    "type": "keyword",
    "index": true,
    "store": false,
    "sortable": true,
    "aggregatable": true,
    "highlightable": false
    },
    "model": {
    "type": "keyword",
    "store": true,
    "sortable": false,
    "aggregatable": true,
    "highlightable": false
    },
    "text": {
    "type": "text",
    "store": true,
    "sortable": false,
    "aggregatable": true,
    "highlightable": true
    }
    }
    }

语法

判空

len(result.Translations) != 0
// 等效于
result.Translations != nil && result.Translations.Text != ""

slice-map-init

var sli []string
// 正常, append可以初始化
sli = append(sli, "a")

var map map[string]string
// 会报错空指针
map["a"] = "a"

var map = make(map[string]string)
// 正常
map["a"] = "a"

mongo

insertMany

ordered=false 可以忽视插入失败的, 能正常插入的不回退

opts := options.InsertMany().SetOrdered(false)
result, err := d.t.InsertMany(ctx, ps, opts)

mq

权限及连接

  • aliyun-mq 内网/外网为两个地址+账号

    internal url +内网账号

    external url + 外网账号

没创建 mq 或者账号对应错误就会出现下面报错

GID_SIMSWAP_SPLIT_TEST_JOB&ns=MQ_INST_1195962631068374_BYL49Q98&numOfMessages=1&waitseconds=30, requestId: 6371A9653942330E0073F9D1, Id: MQ#101:83F0859
[E]2022-11-14 10:35:18.041: failed to receive message: aliyun_mq response status error,code: AccessDenied, message: valid resource owner failed. maybe the resource MQ_INST_1195962631068374_BYL49Q98%simswap_split_test_job not created, path: topics/simswap_split_test_job/messages?consumer=GID_SIMSWAP_SPLIT_TEST_JOB&ns=MQ_INST_1195962631068374_BYL49Q98&numOfMessages=1&waitseconds=30, requestId: 6371A9663942330E0073FCB6, Id: MQ#101:90146AF
[E]2022-11-14 10:35:19.044: failed to receive message: aliyun_mq response status error,code: AccessDenied, message: valid resource owner failed. maybe the resource MQ_INST_1195962631068374_BYL49Q98%simswap_split_test_job not created, path: topics/simswap_split_test_job/messages?consumer=GID_SIMSWAP_SPLIT_TEST_JOB&ns=MQ_INST_1195962631068374_BYL49Q98&numOfMessages=1&waitseconds=30, requestId: 6371A9673942330E0073FF50, Id: MQ#101:D8A6C84
[E]2022-11-14 10:35:20.047: failed to receive message: aliyun_mq response status error,code: AccessDenied, message: valid resource owner failed. maybe the resource MQ_INST_1195962631068374_BYL49Q98%simswap_split_test_job not created, path: topics/simswap_split_test_job/messages?consumer=GID_SIMSWAP_SPLIT_TEST_JOB&ns=MQ_INST_1195962631068374_BYL49Q98&numOfMessages=1&waitseconds=30, requestId: 6371A9683942330E0074020B, Id: MQ#101:7D60BBD


aliyun_mq response status error,code: AccessDenied, message: valid resource owner failed. maybe the resource MQ_INST_1195962631068374_BYL49Q98%simswap_split_job_test not created, path: topics/simswap_split_job_test/messages?ns=MQ_INST_1195962631068374_BYL49Q98, requestId: 63721BA23842370E005637D5, Id: MQ#101:117860E

分割线

常见问题

阿里云效-go-mod

官方文档

代码服务常见问题 FAQ


私有库连接问题

私有库无论走代理/Direct 都拉不到, 只需要配置下面这个

# 代理黑名单 - 不走代理的域名; 公司用的私有库走代理是找不找的, 需要写进黑名单; 一般只需要改 GOPRIVATE, 后两个默认取 GOPRIVATE 所以不需要动
[Environment]::SetEnvironmentVariable('GOPRIVATE', 'codeup.aliyun.com', 'User')
[Environment]::SetEnvironmentVariable('GONOPROXY', 'codeup.aliyun.com', 'User')
[Environment]::SetEnvironmentVariable('GONOSUMDB', 'codeup.aliyun.com', 'User')

# 建议用下面这个命令
go env -w GOPRIVATE=codeup.aliyun.com

鉴权失败问题

通过 go get 下载云效的私有 Mod 时需要做鉴权

在用户目录 ~/ 下新建 .netrc 或者 _netrc (对应 Linux/Mac 与 Windows), 添加如下内容

machine codeup.aliyun.com
login xxx
password xxxx

注意这里的用户名/密码是云效->个人设置->HTTPS 密码, 不是登录云效用的用户名/密码, 否则会出现以下报错 (环境变量/文件都配置对了但鉴权失败)

Error: unrecognized import path “codeup.aliyun.com/wenuts/basis/pollen”: parse https://codeup.aliyun.com/wenuts/basis/pollen?go-get=1: no go-import meta tags ()

<!doctype html>
<html>
<head>
<meta message='鉴权失败, 请确认客户端是否配置&quot;.netrc&quot;文件,traceId:707c9fc316602743714102416e4ef0'>
</head>
</html>

Macos

go: codeup.aliyun.com/wenuts/basis/gox@v0.0.0-20220802063744-4facf115cdc5: invalid version: git ls-remote -q origin in /Users/weidows/go/pkg/mod/cache/vcs/94a54894325b8a8441b8518aba18b8841ef7b2b555fe37fcd220c2d9a7755096: exit status 128:
fatal: could not read Username for 'https://codeup.aliyun.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

照搬 windows 上已经配好的配置还是如上报错, 实际上 macos 上还需要在 ~/.gitconfig 里面添加如下 [9]

[url "git@codeup.aliyun.com:"]
insteadOf = https://codeup.aliyun.com/

GOSUMDB-代理问题

╰─ go get -u codeup.aliyun.com/wenuts/aimage/service
go: golang.org/x/crypto@v0.0.0-20210817164053-32db794688a5: verifying go.mod: invalid GOSUMDB: malformed verifier id

有时 GOSUMDB 设置代理会出现上面问题, 只需要把 GOSUMDB 置空就行:

go env -w GOSUMDB=

hostname-无法解析

ssh: Could not resolve hostname codeup.aliyun.com: Name or service not known
Ping request could not find host codeup.aliyun.com. Please check the name and try again.
或者
ssh: Could not resolve hostname codeup.aliyun.com: Name or service not known Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.

很奇葩的问题, 通过 ssh 推代码或者 ping 时都不通, 但是网页可以访问, 清理本机 DNS 缓存也不管用

直到换了个 WIFI 试了试…

分割线

protobuf-无法获取

手动下载的话可以参照: [8] 另外配置好下面提到的配置+有魔法能力, 不会出现这问题

go env -w GOPROXY=https://proxy.golang.com.cn,direct
go env -w GOSUMDB=

package

missing go.sum entry for module providing package

就如报错信息一样, 缺少 go.sum, 执行 go mod tidy

go install github.com/Weidows/Golang/cmd/common-starter@latest
but does not contain package github.com/Weidows/Golang/cmd/common-starter

去掉尾部的@latest


go-get-更新依赖无效

  1. 很有可能因为设置了 GOPROXY, 拉不到最新的所以无效, 换成默认的就行了:

    go env -w GOPROXY=
  2. cannot find module providing package xxx

    很怪的问题, 全更新就解决了

    go get -u all

cgo

av_read_frame failed and ret:-541478725

有可能只是单纯读帧读完了, 实际是正常退出


流没关闭导致读取失败

有业务: 拿到图片文件写入到 oss,并把 oss 图片地址返回前端展示

但是出现问题: 文件写入后把正常地址传给前端, 前端获取图片, code200 但是没数据

也就是前端从 oss 获取图片时, golang 还占着文件没放, 导致前端能找到图片但读不到数据


类型参数-主类型不一致

annot range over array (variable of type S constrained by interface{[]any|map[any]any}) (S has no core type)

func ForEach[S []any | map[any]any](array S, fn func(index int, value any)) {
for i, v := range array {
fn(i, v)
}
}

[]any 与 map[any]any 是不同的主类型, 虽然不会报错但是编译通不过, 改为 []any | any 就不会报错了 [11]

Access-is-denied

• Compiling application: go: error obtaining buildID for go tool compile: fork/exec D:\Scoop\apps\gvm\current.g\go\pkg\tool\windows_amd64\compile.exe: Access is denied.

很有可能是符号链接导致的, 套了不止一层就会出现这个问题

分割线

借物表

[1]: 《Golang 区块链入门到实战_以太坊/fabric》

[2]: 私有模块

[3]: go mod init 在初始化时出现 cannot determine module path for source directory (outside GOPATH

[5]: https://www.runoob.com/go/go-tutorial.html

[6]: go 语言入门: package command-line-arguments is not a main package

[7]: golang 之 import 和 package 的使用

[8]: 如何解决 Golang 获取 google.golang.org/protobuf 包报错的问题

[9]: 使用 go get 或 go mod download 在私人儲存庫遇到 fatal: could not read Username for ‘https://github.com’: terminal prompts disabled

[10]: LD_LIBRARY_PATH 与-L 的关系以及延伸_51CTO 博客_LD_LIBRARY_PATH

[11]: Go 泛型系列:Go1.18 类型约束那些事-技术圈