cubesandbox创建模版报错
模板构建失败。
该模板构建失败，无法查看详情，但你可以将其删除
CubeSandbox 模板构建失败（FAILED），页面只显示：
模板构建失败。
该模板构建失败，无法查看详情
Job ID:
ceaa39fc-aa7b-43d4-974a-1ef430a074f7
这里需要从 CubeSandbox 后端 Job 日志定位，前端不会展示具体原因
结合你的环境（v0.7.1 + PVM + MinIO + Harbor 镜像），优先排查以下几个方向
1. 根据 Job ID 查构建日志（第一步）
你的 Job ID：
ceaa39fc-aa7b-43d4-974a-1ef430a074f7
在 CubeSandbox 节点执行：
grep -R "ceaa39fc-aa7b-43d4-974a-1ef430a074f7" /var/log 2>/dev/null
或者：
journalctl -xe | grep ceaa39fc
如果 CubeSandbox 是容器：
docker ps | grep cube
然后：
docker logs <cube-container> | grep ceaa39fc
2. 查看 CubeSandbox 服务日志
先找服务：
systemctl list-units | grep cube
常见：
cube-sandbox.service
cube-server.service
cube-agent.service
查看：
journalctl -u cube-sandbox -n 200 --no-pager
重点搜索：
journalctl -u cube-sandbox -n 500 | egrep "ERROR|failed|ceaa39fc"
3. 你的截图里有一个重要信息
模板来源：
镜像:
harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandb...
说明模板创建流程大概率：
Harbor 镜像
|
v
CubeSandbox Builder
|
v
生成 CFS/MinIO 快照
|
v
Template
所以重点检查 Harbor 拉取
4. 检查 Harbor 镜像访问
在 CubeSandbox 主机：
docker login harbor-intranet.allintechinc.com
测试：
docker pull harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:xxx
如果失败：
常见：
认证失败
unauthorized: authentication required
需要配置 Harbor 凭证
TLS问题
例如：
x509: certificate signed by unknown authority
需要导入 CA：
/etc/docker/certs.d/harbor-intranet.allintechinc.com/ca.crt
5. 检查 MinIO/CFS
你安装时：
CUBE_S3_ENDPOINT=http://192.168.0.6:9000
bucket=cube-volumes
模板生成需要写入：
mc ls cube/cube-volumes
检查空间：
df -h
重点：
/var
/tmp
/mnt
模板构建可能需要几十 GB 临时空间
6. 检查 PVM/KVM
你的安装：
PVM host detected
kvm_pvm loaded
检查：
lsmod | grep kvm
应该：
kvm_pvm
kvm
检查：
/dev/kvm
权限：
ls -l /dev/kvm
7. 删除失败模板
页面删除失败模板即可
如果 UI 删除失败：
查模板：
kubectl get template -A
或者：
docker exec -it <cube-container> sh
查看数据库：
grep -R "tpl-464f1b5538764bd7afa240ca" /
8. 建议你现在执行这几个命令
按顺序：
① 找 Job
grep -R "ceaa39fc-aa7b-43d4-974a-1ef430a074f7" /var/log 2>/dev/null
② 看服务
systemctl list-units | grep cube
③ 看最近错误
journalctl -xe -n 200 | grep -i cube
④ 看磁盘
df -h
[root@VM-0-6-tencentos ~]# systemctl list-units | grep -i cube sys-devices-virtual-net-cube\x2ddev.device loaded active plugged /sys/devices/virtual/net/cube-dev sys-devices-virtual-net-cube\x2ddns0.device loaded active plugged /sys/devices/virtual/net/cube-dns0 sys-subsystem-net-devices-cube\x2ddev.device loaded active plugged /sys/subsystem/net/devices/cube-dev sys-subsystem-net-devices-cube\x2ddns0.device loaded active plugged /sys/subsystem/net/devices/cube-dns0 usr-local-services-cubetoolbox-cubeletmnt-mnt.mount loaded active mounted /usr/local/services/cubetoolbox/cubeletmnt/mnt usr-local-services-cubetoolbox-cubeletmnt.mount loaded active mounted /usr/local/services/cubetoolbox/cubeletmnt cube-sandbox-coredns.service loaded active running Cube Sandbox CoreDNS container cube-sandbox-cube-api.service loaded active running Cube Sandbox Cube API cube-sandbox-cube-egress-net.service loaded active exited Cube Sandbox cube-egress host network rules (TPROXY + ip rule) cube-sandbox-cube-egress.service loaded active running Cube Sandbox cube-egress (transparent MITM proxy) cube-sandbox-cube-lifecycle-manager.service loaded active running Cube Sandbox cube-lifecycle-manager container cube-sandbox-cube-proxy.service loaded active running Cube Sandbox cube-proxy container cube-sandbox-cube-templatecenter.service loaded active running Cube Sandbox CubeTemplateCenter cube-sandbox-cubelet.service loaded active running Cube Sandbox Cubelet cube-sandbox-cubemaster.service loaded active running Cube Sandbox CubeMaster cube-sandbox-cubeops.service loaded active running Cube Sandbox CubeOps (admin/ops API) cube-sandbox-dns.service loaded active exited Cube Sandbox host DNS routing cube-sandbox-minio.service loaded active running Cube Sandbox MinIO container cube-sandbox-mysql.service loaded active running Cube Sandbox MySQL container cube-sandbox-redis.service loaded active running Cube Sandbox Redis container cube-sandbox-webui.service loaded active running Cube Sandbox Web UI container cube-sandbox-control.target loaded active active Cube Sandbox control-plane stack [root@VM-0-6-tencentos ~]# journalctl -xe -n 300 | grep -Ei "cube|template|job|error|failed" 9月 20 17:47:44 VM-0-6-tencentos bash[5538]: [0.450ms] [rows:0] SELECT * FROM
t_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:47:44 VM-0-6-tencentos d5c0c2c48c92[1915]: 1.119.147.226 - - [20/Sep/2026:09:47:44 +0000] "GET /templates/tpl-464f1b5538764bd7afa240ca HTTP/1.1" 404 61 "http://152.136.174.29:12088/templates/tpl-464f1b5538764bd7afa240ca" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36" 9月 20 17:47:44 VM-0-6-tencentos bash[6300]: cube-webui | 1.119.147.226 - - [20/Sep/2026:09:47:44 +0000] "GET /templates/tpl-464f1b5538764bd7afa240ca HTTP/1.1" 404 61 "http://152.136.174.29:12088/templates/tpl-464f1b5538764bd7afa240ca" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36" 9月 20 17:47:44 VM-0-6-tencentos d5c0c2c48c92[1915]: 1.119.147.226 - - [20/Sep/2026:09:47:44 +0000] "GET /templates/compat HTTP/1.1" 200 112 "http://152.136.174.29:12088/templates/tpl-464f1b5538764bd7afa240ca" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36" 9月 20 17:47:44 VM-0-6-tencentos bash[6300]: cube-webui | 1.119.147.226 - - [20/Sep/2026:09:47:44 +0000] "GET /templates/compat HTTP/1.1" 200 112 "http://152.136.174.29:12088/templates/tpl-464f1b5538764bd7afa240ca" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36" 9月 20 17:47:45 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:47:45 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:47:45 VM-0-6-tencentos bash[5538]: 2026/09/20 17:47:45 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:47:45 VM-0-6-tencentos bash[5538]: [0.683ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:47:45 VM-0-6-tencentos bash[5538]: 2026/09/20 17:47:45 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:47:45 VM-0-6-tencentos bash[5538]: [0.397ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:47:45 VM-0-6-tencentos d5c0c2c48c92[1915]: 1.119.147.226 - - [20/Sep/2026:09:47:45 +0000] "GET /templates/tpl-464f1b5538764bd7afa240ca HTTP/1.1" 404 61 "http://152.136.174.29:12088/templates/tpl-464f1b5538764bd7afa240ca" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36" 9月 20 17:47:45 VM-0-6-tencentos bash[6300]: cube-webui | 1.119.147.226 - - [20/Sep/2026:09:47:45 +0000] "GET /templates/tpl-464f1b5538764bd7afa240ca HTTP/1.1" 404 61 "http://152.136.174.29:12088/templates/tpl-464f1b5538764bd7afa240ca" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36" 9月 20 17:47:55 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:47:55 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:48:05 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:48:05 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:48:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:48:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:48:12 VM-0-6-tencentos bash[5538]: [0.411ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:48:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:48:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:48:12 VM-0-6-tencentos bash[5538]: [0.387ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:48:15 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:48:15 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:48:25 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:48:25 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:48:35 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:48:35 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:48:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:48:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:48:42 VM-0-6-tencentos bash[5538]: [0.371ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:48:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:48:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:48:42 VM-0-6-tencentos bash[5538]: [0.363ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:48:45 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:48:45 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:48:55 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:48:55 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:49:05 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:49:05 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:49:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:49:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:49:12 VM-0-6-tencentos bash[5538]: [0.361ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:49:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:49:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:49:12 VM-0-6-tencentos bash[5538]: [0.356ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:49:15 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:49:15 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:49:25 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:49:25 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:49:35 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:49:35 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:49:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:49:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:49:42 VM-0-6-tencentos bash[5538]: [0.426ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:49:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:49:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:49:42 VM-0-6-tencentos bash[5538]: [0.403ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:49:45 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:49:45 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:49:54 VM-0-6-tencentos bash[3723]: cube-sandbox-redis | 1:M 20 Sep 2026 09:49:54.070 * 100 changes in 300 seconds. Saving... 9月 20 17:49:54 VM-0-6-tencentos bash[3723]: cube-sandbox-redis | 1:M 20 Sep 2026 09:49:54.070 * Background saving started by pid 13477 9月 20 17:49:54 VM-0-6-tencentos bash[3723]: cube-sandbox-redis | 13477:C 20 Sep 2026 09:49:54.073 * DB saved on disk 9月 20 17:49:54 VM-0-6-tencentos bash[3723]: cube-sandbox-redis | 13477:C 20 Sep 2026 09:49:54.073 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB 9月 20 17:49:54 VM-0-6-tencentos bash[3723]: cube-sandbox-redis | 1:M 20 Sep 2026 09:49:54.171 * Background saving terminated with success 9月 20 17:49:56 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:49:56 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:50:06 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:50:06 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:50:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:50:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:50:12 VM-0-6-tencentos bash[5538]: [0.374ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:50:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:50:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:50:12 VM-0-6-tencentos bash[5538]: [0.366ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:50:16 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:50:16 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:50:26 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:50:26 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:50:36 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:50:36 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:50:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:50:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:50:42 VM-0-6-tencentos bash[5538]: [0.350ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:50:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:50:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:50:42 VM-0-6-tencentos bash[5538]: [0.329ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:50:46 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:50:46 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:50:56 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:50:56 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:51:06 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:51:06 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:51:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:51:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:51:12 VM-0-6-tencentos bash[5538]: [0.421ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:51:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:51:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:51:12 VM-0-6-tencentos bash[5538]: [0.366ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:51:16 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:51:16 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:51:26 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:51:26 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:51:36 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:51:36 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:51:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:51:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:51:42 VM-0-6-tencentos bash[5538]: [0.386ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:51:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:51:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:51:42 VM-0-6-tencentos bash[5538]: [0.384ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:51:46 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:51:46 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:51:56 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:51:56 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:52:06 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:52:06 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:52:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:52:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:52:12 VM-0-6-tencentos bash[5538]: [0.369ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:52:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:52:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:52:12 VM-0-6-tencentos bash[5538]: [0.375ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:52:16 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:52:16 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:52:26 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:52:26 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:52:36 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:52:36 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:52:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:52:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:52:42 VM-0-6-tencentos bash[5538]: [0.341ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:52:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:52:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:52:42 VM-0-6-tencentos bash[5538]: [0.325ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:52:46 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:52:46 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:52:56 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:52:56 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:53:06 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:53:06 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:53:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:53:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:53:12 VM-0-6-tencentos bash[5538]: [0.368ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:53:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:53:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:53:12 VM-0-6-tencentos bash[5538]: [0.382ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:53:16 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:53:16 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" 9月 20 17:53:26 VM-0-6-tencentos bash[6300]: cube-webui | 127.0.0.1 - - [20/Sep/2026:09:53:26 +0000] "GET / HTTP/1.1" 200 1120 "-" "Wget/1.21.3" [root@VM-0-6-tencentos ~]# docker ps | grep cube 5d9ef03bf1ed cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-egress:v0.7.1 "/usr/local/openrest…" About an hour ago Up About an hour cube-egress d5c0c2c48c92 cube-sandbox-image.tencentcloudcr.com/opensource/openresty:1.21.4.1-6-alpine-fat "/usr/local/openrest…" About an hour ago Up About an hour (healthy) 0.0.0.0:12088->80/tcp cube-webui 5041bbb48a0f cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-proxy:v0.7.1 "/usr/local/openrest…" About an hour ago Up About an hour cube-proxy 459a081ffa37 cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-lifecycle-manager:v0.7.1 "/usr/local/bin/cube…" About an hour ago Up About an hour cube-lifecycle-manager 8d12b0ccbb0b cube-sandbox-image.tencentcloudcr.com/opensource/redis:7-alpine "docker-entrypoint.s…" About an hour ago Up About an hour (healthy) 127.0.0.1:6379->6379/tcp cube-sandbox-redis 76096c049704 cube-sandbox-image.tencentcloudcr.com/opensource/mysql:8.0 "docker-entrypoint.s…" About an hour ago Up About an hour (healthy) 127.0.0.1:3306->3306/tcp, 33060/tcp cube-sandbox-mysql 861435ecd634 cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/minio:RELEASE.2025-09-07T16-13-09Z "/usr/bin/docker-ent…" About an hour ago Up About an hour (healthy) 192.168.0.6:9000->9000/tcp, 127.0.0.1:9001->9001/tcp cube-sandbox-minio a0dd3c78f388 cube-sandbox-image.tencentcloudcr.com/opensource/coredns/coredns:1.14.2 "/coredns -conf /etc…" About an hour ago Up About an hour cube-proxy-coredns [root@VM-0-6-tencentos ~]# grep -R "ceaa39fc-aa7b-43d4-974a-1ef430a074f7" /var/log 2>/dev/null [root@VM-0-6-tencentos ~]#
根据贴出的日志，已经可以缩小范围：
CubeSandbox 控制面正常运行，但模板构建任务记录已经丢失，没有留下失败原因
关键日志：
SELECT * FROM `t_cube_template_definition`
WHERE template_id = 'tpl-464f1b5538764bd7afa240ca'
...
record not found
以及：
SELECT * FROM `t_cube_snapshot`
WHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca'
...
record not found
说明：
- WebUI 请求模板详情
- CubeMaster 查询模板定义
- MySQL 中已经没有该模板记录
- snapshot 记录也不存在
所以现在看到的：
模板构建失败
无法查看详情
不是当前实时错误，而是构建任务失败后清理了数据库记录
当前重点：找真正的 builder 日志
你的 systemd 列表里有：
cube-sandbox-cubelet.service
cube-sandbox-cubemaster.service
cube-sandbox-cube-templatecenter.service
cube-sandbox-cube-api.service
cube-sandbox-cubeops.service
模板构建通常不是 WebUI/CubeMaster 报错，而是在：
- cubelet
- lifecycle-manager
- templatecenter
里面
1. 查看 cubelet 日志（优先）
执行：
journalctl -u cube-sandbox-cubelet -n 500 --no-pager
过滤：
journalctl -u cube-sandbox-cubelet --no-pager | \
grep -Ei "template|snapshot|build|error|failed|harbor|image"
2. 查看 lifecycle-manager
模板创建任务通常由它执行：
journalctl -u cube-sandbox-cube-lifecycle-manager -n 500 --no-pager
过滤：
journalctl -u cube-sandbox-cube-lifecycle-manager --no-pager | \
grep -Ei "template|snapshot|error|failed"
3. 查看 TemplateCenter
journalctl -u cube-sandbox-cube-templatecenter -n 500 --no-pager
4. Docker 容器日志
你的容器：
459a081ffa37 cube-lifecycle-manager
查看：
docker logs 459a081ffa37 --tail 500
重点：
docker logs 459a081ffa37 2>&1 | \
grep -Ei "template|snapshot|error|failed"
5. 另外发现一个可能问题
你的模板来源之前截图：
harbor-intranet.allintechinc.com/llm-planning/...
但当前：
docker ps
没有看到任何 builder 容器
正常模板构建期间应该可能出现：
- sandbox builder
- qemu
- guest
- converter
现在全部没有
可能情况：
情况A
构建任务启动后立即失败
常见原因：
- Harbor 拉取失败
- 镜像权限失败
- 镜像格式不支持
情况B
PVM 创建失败
你之前安装：
CUBE_PVM_ENABLE=1
重点检查：
lsmod | grep kvm
和：
ls -l /dev/kvm
6. 检查 MinIO 是否有模板残留
因为模板 snapshot 存 MinIO：
docker exec -it cube-sandbox-minio sh
或者：
mc alias ls
mc ls <alias>/cube-volumes
看看是否有：
tpl-464f1b5538764bd7afa240ca
7. 建议下一步直接执行这几个
一次贴回来：
journalctl -u cube-sandbox-cubelet -n 200 --no-pager
journalctl -u cube-sandbox-cube-lifecycle-manager -n 200 --no-pager
docker logs 459a081ffa37 --tail 200
我预计错误会在这里出现
目前可以排除：
✅ CubeSandbox 主服务启动问题
✅ MySQL/Redis异常
✅ WebUI异常
重点怀疑：
- Harbor 镜像拉取失败
- PVM guest 创建失败
- 模板 snapshot 生成失败
- MinIO 写入失败
下一批日志基本可以直接定位。
继续排查。你当前贴的日志已经证明：
cube-sandbox-*控制面正常 ✅- MySQL/Redis/MinIO 容器正常运行 ✅
- 失败模板的数据库记录已经被清理 ❗
- 真正失败点发生在模板构建执行阶段，不在 WebUI
下一步需要抓 构建执行组件日志。
请执行下面命令：
1. Cubelet（最重要）
journalctl -u cube-sandbox-cubelet -n 500 --no-pager
重点看：
template
snapshot
pull
image
qemu
kvm
mount
error
failed
也可以过滤：
journalctl -u cube-sandbox-cubelet --no-pager | \
grep -Ei "template|snapshot|image|pull|qemu|kvm|error|failed"
2. Lifecycle Manager
模板生命周期一般由它调度：
journalctl -u cube-sandbox-cube-lifecycle-manager -n 500 --no-pager
过滤：
journalctl -u cube-sandbox-cube-lifecycle-manager --no-pager | \
grep -Ei "template|snapshot|build|error|failed"
3. TemplateCenter
journalctl -u cube-sandbox-cube-templatecenter -n 300 --no-pager
4. 查看是否有临时构建容器
模板失败后可能已经退出：
docker ps -a | grep -Ei "cube|template|builder|qemu"
5. 检查 PVM 环境
因为你安装用了：
CUBE_PVM_ENABLE=1
执行：
lsmod | grep kvm
以及：
ls -l /dev/kvm
正常应该类似：
/dev/kvm
6. 检查 Harbor 连通性
你的模板镜像来自：
harbor-intranet.allintechinc.com/llm-planning/...
执行：
curl -k https://harbor-intranet.allintechinc.com/v2/
正常：
{}
如果：
401 Unauthorized
说明需要配置 Harbor 登录
我目前最怀疑的两个点
可能1：Harbor 私有镜像无法拉取
流程：
创建模板
↓
CubeSandbox
↓
pull Harbor image
↓
失败
↓
删除 template/snapshot记录
符合你现在现象
可能2：PVM 模板转换失败
因为：
CUBE_PVM_ENABLE=1
如果：
- kernel 不匹配
- /dev/kvm 不可用
- guest kernel 创建失败
也会出现模板直接失败
journalctl -u cube-sandbox-cubelet -n 200 --no-pager journalctl -u cube-sandbox-cube-lifecycle-manager -n 200 --no-pager docker ps -a | grep -Ei "cube|template|builder|qemu" 9月 20 15:36:05 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cubelet.service - Cube Sandbox Cubelet... 9月 20 15:36:05 VM-0-6-tencentos bash[5832]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 15:36:05 VM-0-6-tencentos bash[5922]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 15:36:06 VM-0-6-tencentos bash[5875]: [one-click-systemd] updated cubelet cow.s3 enable=false 9月 20 15:36:06 VM-0-6-tencentos bash[5875]: [one-click-systemd] cubelet initial pid=6023 9月 20 15:36:06 VM-0-6-tencentos bash[7079]: 2026-09-20T15:36:06.584521396+08:00,listen file[/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml] 9月 20 15:36:06 VM-0-6-tencentos bash[7079]: cfg:{"Common":{"CommonTimeout":10000000000,"LogLevel":"","DescribeAsyncInterval":100000000,"EnablePFMode":false,"DescribeBDFInterval":10000000,"DescribeBDFTimeout":10000000000,"GetBDFByUuidCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_uuid","GetBDFByIfNameCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_ifname","CommandTimeout":1000000000,"DisableHostCgroup":true,"DisableVmCgroup":false,"EnableSandboxExecCmdBeforeExist":false,"SandboxExecCmdTimeOut":5000000000,"SandboxExecCmdBeforeExist":["sh","/usr/local/cubetools/cube_exec_cmd_before_exist.sh"],"SandboxExecCmdBeforeExistLogOut":false,"SandboxExecCmdOutMaxLines":2000,"SandboxExecCmdMatchLine":"","SandboxExecCmdAfterMatch":null,"SandboxExecCmdBase64AfterMatch":"","SandboxExecCmdAfterMatchLogOut":false,"CgroupDisableMemoryReparentFile":"","CgroupDisableCpusetList":"","DisableHostNetfile":true,"DefaultDNSServers":["119.29.29.29"],"DefaultDNSSearches":[],"DefaultDNSOptions":[],"ReconcileInterval":300000000000,"DisableCubeBoxTemplateBaseFormatPoolOfNumberVer":false},"Tenant":{"Tenants":{},"UseRespectiveNamespace":false,"DefaultNS":"default","ZiyanUinListStr":""},"MetaServerConfig":{"MetaServerEndpoint":"127.0.0.1:3010","NodeStatusMaxImages":40000,"CubeMasterHTTPAddr":"127.0.0.1:8089"},"HostConf":{"Quota":{"Cpu":0,"Mem":"","MvmLimit":0,"CreationConcurrentNum":0,"PausedResourceReleaseRatio":0},"GC":{"CodeExpirationTime":"72h","ImageExpirationTime":"24h"},"SchedulerLabel":"default-cluster"}} 9月 20 15:36:06 VM-0-6-tencentos bash[7079]: 2026/09/20 15:36:06 mounted state tmpfs /data/cubelet/state size=1024m 9月 20 15:36:06 VM-0-6-tencentos bash[7079]: 2026/09/20 15:36:06 set /data/cubelet/root as shared mount 9月 20 15:36:07 VM-0-6-tencentos bash[5875]: /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh: line 57: 6023 Killed ( export CUBE_SANDBOX_NODE_IP="${CUBE_SANDBOX_NODE_IP}"; exec "${CUBELET_BIN}" --config "${CUBELET_CONFIG}" --dynamic-conf-path "${CUBELET_DYNAMICCONF}" ) 9月 20 15:36:07 VM-0-6-tencentos bash[5875]: [one-click-systemd] cubelet stable pid=7079 9月 20 15:36:07 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cubelet.service - Cube Sandbox Cubelet. 9月 20 16:18:54 VM-0-6-tencentos systemd[1]: Stopping cube-sandbox-cubelet.service - Cube Sandbox Cubelet... 9月 20 16:18:55 VM-0-6-tencentos bash[87481]: [one-click-systemd] cubelet stopped pid=7079 9月 20 16:18:55 VM-0-6-tencentos systemd[1]: cube-sandbox-cubelet.service: Deactivated successfully. 9月 20 16:18:55 VM-0-6-tencentos systemd[1]: Stopped cube-sandbox-cubelet.service - Cube Sandbox Cubelet. 9月 20 16:18:55 VM-0-6-tencentos systemd[1]: cube-sandbox-cubelet.service: Consumed 10.594s CPU time, 237.9M memory peak, 0B memory swap peak. -- Boot f6f2fcd1381547d990b4d2b666efa29d -- 9月 20 16:19:13 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cubelet.service - Cube Sandbox Cubelet... 9月 20 16:19:13 VM-0-6-tencentos bash[1510]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 16:19:13 VM-0-6-tencentos bash[1587]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 16:19:13 VM-0-6-tencentos bash[1566]: [one-click-systemd] updated cubelet cow.s3 enable=false 9月 20 16:19:13 VM-0-6-tencentos bash[1566]: [one-click-systemd] cubelet initial pid=1668 9月 20 16:19:14 VM-0-6-tencentos bash[2125]: 2026-09-20T16:19:14.223712774+08:00,listen file[/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml] 9月 20 16:19:14 VM-0-6-tencentos bash[2125]: cfg:{"Common":{"CommonTimeout":10000000000,"LogLevel":"","DescribeAsyncInterval":100000000,"EnablePFMode":false,"DescribeBDFInterval":10000000,"DescribeBDFTimeout":10000000000,"GetBDFByUuidCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_uuid","GetBDFByIfNameCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_ifname","CommandTimeout":1000000000,"DisableHostCgroup":true,"DisableVmCgroup":false,"EnableSandboxExecCmdBeforeExist":false,"SandboxExecCmdTimeOut":5000000000,"SandboxExecCmdBeforeExist":["sh","/usr/local/cubetools/cube_exec_cmd_before_exist.sh"],"SandboxExecCmdBeforeExistLogOut":false,"SandboxExecCmdOutMaxLines":2000,"SandboxExecCmdMatchLine":"","SandboxExecCmdAfterMatch":null,"SandboxExecCmdBase64AfterMatch":"","SandboxExecCmdAfterMatchLogOut":false,"CgroupDisableMemoryReparentFile":"","CgroupDisableCpusetList":"","DisableHostNetfile":true,"DefaultDNSServers":["119.29.29.29"],"DefaultDNSSearches":[],"DefaultDNSOptions":[],"ReconcileInterval":300000000000,"DisableCubeBoxTemplateBaseFormatPoolOfNumberVer":false},"Tenant":{"Tenants":{},"UseRespectiveNamespace":false,"DefaultNS":"default","ZiyanUinListStr":""},"MetaServerConfig":{"MetaServerEndpoint":"127.0.0.1:3010","NodeStatusMaxImages":40000,"CubeMasterHTTPAddr":"127.0.0.1:8089"},"HostConf":{"Quota":{"Cpu":0,"Mem":"","MvmLimit":0,"CreationConcurrentNum":0,"PausedResourceReleaseRatio":0},"GC":{"CodeExpirationTime":"72h","ImageExpirationTime":"24h"},"SchedulerLabel":"default-cluster"}} 9月 20 16:19:14 VM-0-6-tencentos bash[2125]: 2026/09/20 16:19:14 mounted state tmpfs /data/cubelet/state size=1024m 9月 20 16:19:14 VM-0-6-tencentos bash[2125]: 2026/09/20 16:19:14 set /data/cubelet/root as shared mount 9月 20 16:19:14 VM-0-6-tencentos bash[1566]: /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh: line 57: 1668 Killed ( export CUBE_SANDBOX_NODE_IP="${CUBE_SANDBOX_NODE_IP}"; exec "${CUBELET_BIN}" --config "${CUBELET_CONFIG}" --dynamic-conf-path "${CUBELET_DYNAMICCONF}" ) 9月 20 16:19:14 VM-0-6-tencentos bash[1566]: [one-click-systemd] cubelet stable pid=2125 9月 20 16:19:14 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cubelet.service - Cube Sandbox Cubelet. 9月 20 16:24:13 VM-0-6-tencentos systemd[1]: Stopping cube-sandbox-cubelet.service - Cube Sandbox Cubelet... 9月 20 16:24:14 VM-0-6-tencentos bash[18750]: [one-click-systemd] cubelet stopped pid=2125 9月 20 16:24:14 VM-0-6-tencentos systemd[1]: cube-sandbox-cubelet.service: Deactivated successfully. 9月 20 16:24:14 VM-0-6-tencentos systemd[1]: Stopped cube-sandbox-cubelet.service - Cube Sandbox Cubelet. 9月 20 16:24:14 VM-0-6-tencentos systemd[1]: cube-sandbox-cubelet.service: Consumed 11.631s CPU time. -- Boot 0756528841154b8595d93af011d03619 -- 9月 20 16:24:35 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cubelet.service - Cube Sandbox Cubelet... 9月 20 16:24:35 VM-0-6-tencentos bash[1505]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 16:24:35 VM-0-6-tencentos bash[1608]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 16:24:35 VM-0-6-tencentos bash[1580]: [one-click-systemd] updated cubelet cow.s3 enable=false 9月 20 16:24:35 VM-0-6-tencentos bash[1580]: [one-click-systemd] cubelet initial pid=1710 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026-09-20T16:24:35.552388365+08:00,listen file[/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml] 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: cfg:{"Common":{"CommonTimeout":10000000000,"LogLevel":"","DescribeAsyncInterval":100000000,"EnablePFMode":false,"DescribeBDFInterval":10000000,"DescribeBDFTimeout":10000000000,"GetBDFByUuidCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_uuid","GetBDFByIfNameCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_ifname","CommandTimeout":1000000000,"DisableHostCgroup":true,"DisableVmCgroup":false,"EnableSandboxExecCmdBeforeExist":false,"SandboxExecCmdTimeOut":5000000000,"SandboxExecCmdBeforeExist":["sh","/usr/local/cubetools/cube_exec_cmd_before_exist.sh"],"SandboxExecCmdBeforeExistLogOut":false,"SandboxExecCmdOutMaxLines":2000,"SandboxExecCmdMatchLine":"","SandboxExecCmdAfterMatch":null,"SandboxExecCmdBase64AfterMatch":"","SandboxExecCmdAfterMatchLogOut":false,"CgroupDisableMemoryReparentFile":"","CgroupDisableCpusetList":"","DisableHostNetfile":true,"DefaultDNSServers":["119.29.29.29"],"DefaultDNSSearches":[],"DefaultDNSOptions":[],"ReconcileInterval":300000000000,"DisableCubeBoxTemplateBaseFormatPoolOfNumberVer":false},"Tenant":{"Tenants":{},"UseRespectiveNamespace":false,"DefaultNS":"default","ZiyanUinListStr":""},"MetaServerConfig":{"MetaServerEndpoint":"127.0.0.1:3010","NodeStatusMaxImages":40000,"CubeMasterHTTPAddr":"127.0.0.1:8089"},"HostConf":{"Quota":{"Cpu":0,"Mem":"","MvmLimit":0,"CreationConcurrentNum":0,"PausedResourceReleaseRatio":0},"GC":{"CodeExpirationTime":"72h","ImageExpirationTime":"24h"},"SchedulerLabel":"default-cluster"}} 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026/09/20 16:24:35 mounted state tmpfs /data/cubelet/state size=1024m 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026/09/20 16:24:35 set /data/cubelet/root as shared mount 9月 20 16:24:36 VM-0-6-tencentos bash[1580]: /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh: line 57: 1710 Killed ( export CUBE_SANDBOX_NODE_IP="${CUBE_SANDBOX_NODE_IP}"; exec "${CUBELET_BIN}" --config "${CUBELET_CONFIG}" --dynamic-conf-path "${CUBELET_DYNAMICCONF}" ) 9月 20 16:24:36 VM-0-6-tencentos bash[1580]: [one-click-systemd] cubelet stable pid=1949 9月 20 16:24:36 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cubelet.service - Cube Sandbox Cubelet. 9月 20 15:36:26 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container... 9月 20 15:36:26 VM-0-6-tencentos bash[9297]: [one-click-runtime] cube-lifecycle-manager runtime files prepared under /usr/local/services/cubetoolbox/cube-lifecycle-manager 9月 20 15:36:26 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container. 9月 20 15:36:26 VM-0-6-tencentos bash[9375]: [one-click-runtime] pulling cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-lifecycle-manager:v0.7.1 9月 20 15:36:35 VM-0-6-tencentos bash[11195]: time="2026-09-20T15:36:35+08:00" level=warning msg="/usr/local/services/cubetoolbox/cube-lifecycle-manager/docker-compose.yaml: the attribute
versionis obsolete, it will be ignored, please remove it to avoid potential confusion" 9月 20 15:36:35 VM-0-6-tencentos bash[11195]: Container cube-lifecycle-manager Creating 9月 20 15:36:35 VM-0-6-tencentos bash[11195]: Container cube-lifecycle-manager Created 9月 20 15:36:35 VM-0-6-tencentos bash[11195]: Attaching to cube-lifecycle-manager 9月 20 15:36:35 VM-0-6-tencentos bash[11195]: Container cube-lifecycle-manager Starting 9月 20 15:36:36 VM-0-6-tencentos bash[11195]: cube-lifecycle-manager | {"level":"info","ts":1789889796.0163305,"caller":"cube-lifecycle-manager/main.go:59","msg":"cube-lifecycle-manager starting","redis_addr":"127.0.0.1:6379","cube_proxy_admin_urls":["http://127.0.0.1:8082"],"cubemaster_url":"http://192.168.0.6:8089","listen_addr":"0.0.0.0:8083","instance_identity":"VM-0-6-tencentos","leader_election_enabled":false} 9月 20 15:36:36 VM-0-6-tencentos bash[11195]: cube-lifecycle-manager | {"level":"info","ts":1789889796.0164294,"caller":"cube-lifecycle-manager/main.go:81","msg":"eventbus enabled"} 9月 20 15:36:36 VM-0-6-tencentos bash[11195]: cube-lifecycle-manager | {"level":"info","ts":1789889796.0179007,"caller":"cube-lifecycle-manager/main.go:641","msg":"bootstrap complete","entries":0} 9月 20 15:36:36 VM-0-6-tencentos bash[11195]: cube-lifecycle-manager | {"level":"info","ts":1789889796.0205412,"logger":"http","caller":"httpapi/server.go:86","msg":"cube-lifecycle-manager http server listening","addr":"0.0.0.0:8083"} 9月 20 15:36:36 VM-0-6-tencentos bash[11195]: Container cube-lifecycle-manager Started 9月 20 15:36:48 VM-0-6-tencentos bash[11195]: cube-lifecycle-manager | {"level":"info","ts":1789889808.0200999,"logger":"discovery","caller":"discovery/redis.go:182","msg":"discovery: proxy joined","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082"} 9月 20 15:36:48 VM-0-6-tencentos bash[11195]: cube-lifecycle-manager | {"level":"info","ts":1789889808.0201688,"logger":"replay","caller":"cube-lifecycle-manager/main.go:366","msg":"replay begin","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082","entries":0} 9月 20 15:36:48 VM-0-6-tencentos bash[11195]: cube-lifecycle-manager | {"level":"info","ts":1789889808.0202017,"logger":"replay","caller":"cube-lifecycle-manager/main.go:394","msg":"replay done","proxy_id":"192.168.0.6:8082","pushed":0,"failed":0} 9月 20 16:18:54 VM-0-6-tencentos systemd[1]: Stopping cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container... 9月 20 16:18:54 VM-0-6-tencentos bash[11195]: [45B blob data] 9月 20 16:18:54 VM-0-6-tencentos bash[87476]: [one-click-runtime] cube-lifecycle-manager stopped 9月 20 16:18:54 VM-0-6-tencentos systemd[1]: cube-sandbox-cube-lifecycle-manager.service: Deactivated successfully. 9月 20 16:18:54 VM-0-6-tencentos systemd[1]: Stopped cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container. 9月 20 16:18:54 VM-0-6-tencentos systemd[1]: cube-sandbox-cube-lifecycle-manager.service: Consumed 1.073s CPU time, 29.4M memory peak, 0B memory swap peak. -- Boot f6f2fcd1381547d990b4d2b666efa29d -- 9月 20 16:19:20 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container... 9月 20 16:19:20 VM-0-6-tencentos bash[7085]: [one-click-runtime] cube-lifecycle-manager runtime files prepared under /usr/local/services/cubetoolbox/cube-lifecycle-manager 9月 20 16:19:20 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container. 9月 20 16:19:20 VM-0-6-tencentos bash[7170]: [one-click-runtime] pulling cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-lifecycle-manager:v0.7.1 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: time="2026-09-20T16:19:21+08:00" level=warning msg="/usr/local/services/cubetoolbox/cube-lifecycle-manager/docker-compose.yaml: the attributeversionis obsolete, it will be ignored, please remove it to avoid potential confusion" 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: Container cube-lifecycle-manager Creating 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: Container cube-lifecycle-manager Created 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: Attaching to cube-lifecycle-manager 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: Container cube-lifecycle-manager Starting 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: Container cube-lifecycle-manager Started 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: cube-lifecycle-manager | {"level":"info","ts":1789892361.5976703,"caller":"cube-lifecycle-manager/main.go:59","msg":"cube-lifecycle-manager starting","redis_addr":"127.0.0.1:6379","cube_proxy_admin_urls":["http://127.0.0.1:8082"],"cubemaster_url":"http://192.168.0.6:8089","listen_addr":"0.0.0.0:8083","instance_identity":"VM-0-6-tencentos","leader_election_enabled":false} 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: cube-lifecycle-manager | {"level":"info","ts":1789892361.600487,"caller":"cube-lifecycle-manager/main.go:81","msg":"eventbus enabled"} 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: cube-lifecycle-manager | {"level":"info","ts":1789892361.6083386,"caller":"cube-lifecycle-manager/main.go:641","msg":"bootstrap complete","entries":0} 9月 20 16:19:21 VM-0-6-tencentos bash[7978]: cube-lifecycle-manager | {"level":"info","ts":1789892361.6112247,"logger":"http","caller":"httpapi/server.go:86","msg":"cube-lifecycle-manager http server listening","addr":"0.0.0.0:8083"} 9月 20 16:19:24 VM-0-6-tencentos bash[7978]: cube-lifecycle-manager | {"level":"info","ts":1789892364.61076,"logger":"discovery","caller":"discovery/redis.go:182","msg":"discovery: proxy joined","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082"} 9月 20 16:19:24 VM-0-6-tencentos bash[7978]: cube-lifecycle-manager | {"level":"info","ts":1789892364.6108475,"logger":"replay","caller":"cube-lifecycle-manager/main.go:366","msg":"replay begin","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082","entries":0} 9月 20 16:19:24 VM-0-6-tencentos bash[7978]: cube-lifecycle-manager | {"level":"info","ts":1789892364.610934,"logger":"replay","caller":"cube-lifecycle-manager/main.go:394","msg":"replay done","proxy_id":"192.168.0.6:8082","pushed":0,"failed":0} 9月 20 16:24:13 VM-0-6-tencentos systemd[1]: Stopping cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container... 9月 20 16:24:18 VM-0-6-tencentos bash[7978]: [45B blob data] 9月 20 16:24:18 VM-0-6-tencentos bash[18760]: [one-click-runtime] cube-lifecycle-manager stopped 9月 20 16:24:18 VM-0-6-tencentos systemd[1]: cube-sandbox-cube-lifecycle-manager.service: Deactivated successfully. 9月 20 16:24:18 VM-0-6-tencentos systemd[1]: Stopped cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container. -- Boot 0756528841154b8595d93af011d03619 -- 9月 20 16:24:41 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container... 9月 20 16:24:41 VM-0-6-tencentos bash[4667]: [one-click-runtime] cube-lifecycle-manager runtime files prepared under /usr/local/services/cubetoolbox/cube-lifecycle-manager 9月 20 16:24:41 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cube-lifecycle-manager.service - Cube Sandbox cube-lifecycle-manager container. 9月 20 16:24:41 VM-0-6-tencentos bash[4762]: [one-click-runtime] pulling cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-lifecycle-manager:v0.7.1 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: time="2026-09-20T16:24:42+08:00" level=warning msg="/usr/local/services/cubetoolbox/cube-lifecycle-manager/docker-compose.yaml: the attributeversionis obsolete, it will be ignored, please remove it to avoid potential confusion" 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: Container cube-lifecycle-manager Creating 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: Container cube-lifecycle-manager Created 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: Attaching to cube-lifecycle-manager 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: Container cube-lifecycle-manager Starting 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: Container cube-lifecycle-manager Started 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: cube-lifecycle-manager | {"level":"info","ts":1789892682.4259038,"caller":"cube-lifecycle-manager/main.go:59","msg":"cube-lifecycle-manager starting","redis_addr":"127.0.0.1:6379","cube_proxy_admin_urls":["http://127.0.0.1:8082"],"cubemaster_url":"http://192.168.0.6:8089","listen_addr":"0.0.0.0:8083","instance_identity":"VM-0-6-tencentos","leader_election_enabled":false} 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: cube-lifecycle-manager | {"level":"info","ts":1789892682.428625,"caller":"cube-lifecycle-manager/main.go:81","msg":"eventbus enabled"} 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: cube-lifecycle-manager | {"level":"info","ts":1789892682.4323788,"caller":"cube-lifecycle-manager/main.go:641","msg":"bootstrap complete","entries":0} 9月 20 16:24:42 VM-0-6-tencentos bash[5603]: cube-lifecycle-manager | {"level":"info","ts":1789892682.4353082,"logger":"http","caller":"httpapi/server.go:86","msg":"cube-lifecycle-manager http server listening","addr":"0.0.0.0:8083"} 9月 20 16:24:45 VM-0-6-tencentos bash[5603]: cube-lifecycle-manager | {"level":"info","ts":1789892685.4343956,"logger":"discovery","caller":"discovery/redis.go:182","msg":"discovery: proxy joined","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082"} 9月 20 16:24:45 VM-0-6-tencentos bash[5603]: cube-lifecycle-manager | {"level":"info","ts":1789892685.434483,"logger":"replay","caller":"cube-lifecycle-manager/main.go:366","msg":"replay begin","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082","entries":0} 9月 20 16:24:45 VM-0-6-tencentos bash[5603]: cube-lifecycle-manager | {"level":"info","ts":1789892685.434507,"logger":"replay","caller":"cube-lifecycle-manager/main.go:394","msg":"replay done","proxy_id":"192.168.0.6:8082","pushed":0,"failed":0} 5d9ef03bf1ed cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-egress:v0.7.1 "/usr/local/openrest…" 2 hours ago Up 2 hours cube-egress d5c0c2c48c92 cube-sandbox-image.tencentcloudcr.com/opensource/openresty:1.21.4.1-6-alpine-fat "/usr/local/openrest…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:12088->80/tcp cube-webui 5041bbb48a0f cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-proxy:v0.7.1 "/usr/local/openrest…" 2 hours ago Up 2 hours cube-proxy 459a081ffa37 cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/cube-lifecycle-manager:v0.7.1 "/usr/local/bin/cube…" 2 hours ago Up 2 hours cube-lifecycle-manager 8d12b0ccbb0b cube-sandbox-image.tencentcloudcr.com/opensource/redis:7-alpine "docker-entrypoint.s…" 2 hours ago Up 2 hours (healthy) 127.0.0.1:6379->6379/tcp cube-sandbox-redis 76096c049704 cube-sandbox-image.tencentcloudcr.com/opensource/mysql:8.0 "docker-entrypoint.s…" 2 hours ago Up 2 hours (healthy) 127.0.0.1:3306->3306/tcp, 33060/tcp cube-sandbox-mysql 861435ecd634 cube-sandbox-cn.tencentcloudcr.com/cube-sandbox/minio:RELEASE.2025-09-07T16-13-09Z "/usr/bin/docker-ent…" 2 hours ago Up 2 hours (healthy) 192.168.0.6:9000->9000/tcp, 127.0.0.1:9001->9001/tcp cube-sandbox-minio a0dd3c78f388 cube-sandbox-image.tencentcloudcr.com/opensource/coredns/coredns:1.14.2 "/coredns -conf /etc…" 2 hours ago Up 2 hours cube-proxy-coredns
从日志看，Cube Sandbox v0.7.1 主链路基本启动正常，真正异常点在 cubelet 进程启动阶段被系统 Kill
关键异常：
cubelet initial pid=1710
/usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh: line 57:
1710 Killed
(exec "${CUBELET_BIN}" --config ... )
但是随后：
cubelet stable pid=1949
systemd Started cube-sandbox-cubelet.service
说明：
- wrapper 脚本认为 cubelet 已稳定
- 但是第一次启动的真实 cubelet binary 被杀掉
- 后续可能启动了另一个监听进程/守护逻辑
- systemd 没有认为失败
1. 最大可能原因：OOM Killer 杀掉 cubelet
日志：
Consumed 10.594s CPU time, 237.9M memory peak
虽然 cubelet 自己峰值只有 237MB，但启动瞬间可能：
- 加载模板
- 初始化 qemu
- 扫描 image cache
- 初始化 container runtime
- 读取大量 metadata
触发瞬时内存压力
先确认：
dmesg -T | grep -Ei "killed process|out of memory|oom"
或者：
journalctl -k | grep -Ei "oom|killed"
重点看：
类似：
Out of memory: Killed process 1710 (cubelet)
Killed process 1710 (cubelet) total-vm...
如果有，就是 OOM
2. 检查机器内存
执行：
free -h
以及：
cat /proc/meminfo | head
Cube Sandbox 最低建议：
| 组件 | 建议 |
|---|---|
| cubelet | >=2G |
| cube-proxy | 512M |
| redis/mysql/minio | >=3G |
| 模板构建 | >=4G |
| 运行 sandbox | 8G+ |
如果这是腾讯云轻量机/小规格 VM，很容易发生
3. 检查 systemd 是否限制内存
查看：
systemctl cat cube-sandbox-cubelet
重点：
MemoryLimit=
MemoryMax=
如果存在：
例如：
MemoryMax=256M
会直接 kill
查看：
systemctl show cube-sandbox-cubelet | grep -i memory
4. 检查 cgroup OOM
你的配置里面：
"DisableHostCgroup":true,
"DisableVmCgroup":false
说明：
host cgroup 禁用了
但是 VM cgroup 开启
查看：
cat /sys/fs/cgroup/memory.events
如果看到：
oom 1
oom_kill 1
说明 cgroup kill
5. cubelet binary 单独启动测试
找到 binary：
grep CUBELET_BIN \
/usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh
例如：
CUBELET_BIN=/usr/local/services/cubetoolbox/Cubelet/cubelet
直接运行：
/usr/local/services/cubetoolbox/Cubelet/cubelet \
--config /xxx/conf.yaml \
--dynamic-conf-path /xxx/conf.yaml
观察：
- panic
- segmentation fault
- permission denied
- killed
6. 检查 kernel 是否因为权限杀
执行：
journalctl -k -n 200
关注：
segfault
apparmor
selinux
denied
7. lifecycle-manager 正常
这一部分没有问题：
cube-lifecycle-manager starting
redis_addr:
127.0.0.1:6379
cubemaster_url:
http://192.168.0.6:8089
listen:
0.0.0.0:8083
状态：
bootstrap complete entries=0
http server listening
proxy joined
说明：
✅ redis OK
✅ mysql OK
✅ minio OK
✅ proxy OK
✅ lifecycle manager OK
8. 当前架构状态
现在：
cube-webui
|
cube-proxy
|
cube-lifecycle-manager
|
redis
|
cubelet ❌
|
qemu/container runtime
问题在：
cubelet -> sandbox runtime
9. 建议下一步执行
请贴下面输出：
A. 看 OOM
dmesg -T | grep -Ei "oom|killed"
B. 看机器资源
free -h
df -h
C. 看 cubelet service
systemctl cat cube-sandbox-cubelet
D. 看进程
ps aux | grep cubelet
E. 看 kernel
journalctl -k -n 100 --no-pager
目前判断：
80% 概率是内存/cgroup OOM，15% 是 cubelet 自身启动异常，5% 是权限或内核限制
v0.7.1 其它组件已经健康，先不用动 Redis、MinIO、Proxy。重点查 cubelet 被 Kill 原因
[root@VM-0-6-tencentos ~]# dmesg -T | grep -Ei "oom|killed" [root@VM-0-6-tencentos ~]# free -h df -h total used free shared buff/cache available Mem: 125Gi 9.7Gi 112Gi 9.0Mi 4.0Gi 115Gi Swap: 0B 0B 0B 文件系统 大小 已用 可用 已用% 挂载点 /dev/vda1 50G 15G 36G 29% / devtmpfs 4.0M 0 4.0M 0% /dev tmpfs 63G 24K 63G 1% /dev/shm tmpfs 26G 3.4M 26G 1% /run /dev/vdb 500G 9.7G 491G 2% /mnt/allin_data overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/a0dd3c78f388d00b5da2b45e55232b14edbe6bd76026713aee3d701c19844282 overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/861435ecd634b2db18ec9e127fd778634b7f12f55c52b76dad3185ef796c29fe overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/8d12b0ccbb0b18a4c8ae2bc343c7de840c06e3a4a00ccb67215dd9ab73537407 overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/76096c0497040f2c04de824409505bbb6a89c82c7e4ebdb5e9119c914a0ed519 overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/459a081ffa3731c574f85bdf91424a92f5e703be1dcf7c19f01cb4ba875c82d9 overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/5041bbb48a0fe1c365ccef646224fd7037a04145976ba2feeb04f48ff6742706 overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/d5c0c2c48c9251acf461358177fca6dbafbfc77e8b0f5439703b9bea9d8a8117 overlay 50G 15G 36G 29% /var/lib/docker/rootfs/overlayfs/5d9ef03bf1eda889418b5487f557960fe62caae604871929dcca5a490011e7df 192.168.0.20:/guqqwbm4/ 10G 42M 10G 1% /mnt/allin_data/allin/pod_log/data_share [root@VM-0-6-tencentos ~]# systemctl cat cube-sandbox-cubelet # /etc/systemd/system/cube-sandbox-cubelet.service [Unit] Description=Cube Sandbox Cubelet After=network-online.target Wants=network-online.target ConditionPathExists=/usr/local/services/cubetoolbox/.one-click.env RequiresMountsFor=/data/cubelet /data/cube-shim /data/snapshot_pack PartOf=cube-sandbox-control.target cube-sandbox-compute.target [Service] Type=forking ExecStartPre=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/prepare-compute-role.sh User=root WorkingDirectory=/usr/local/services/cubetoolbox EnvironmentFile=-/usr/local/services/cubetoolbox/.one-click.env PIDFile=/run/cube-sandbox-systemd/cubelet.pid Delegate=yes KillMode=process ExecStart=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh ExecStop=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubelet-stop.sh Restart=on-failure RestartSec=2s TimeoutStartSec=60s TimeoutStopSec=30s [Install] WantedBy=cube-sandbox-control.target WantedBy=cube-sandbox-compute.target [root@VM-0-6-tencentos ~]# ps aux | grep cubelet root 1949 0.5 0.1 4164304 221340 ? Sl 16:24 0:31 /usr/local/services/cubetoolbox/Cubelet/bin/cubelet --config /usr/local/services/cubetoolbox/Cubelet/config/config.toml --dynamic-conf-path /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml root 178554 0.0 0.0 6696 2060 pts/0 S+ 18:03 0:00 grep --color=auto cubelet [root@VM-0-6-tencentos ~]# journalctl -k -n 100 --no-pager 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: auth-rpcgss-module.service - Kernel Module supporting RPCSEC_GSS was skipped because of an unmet condition check (ConditionPathExists=/etc/krb5.keytab). 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: loadmodules.service - Load legacy module configuration was skipped because no trigger condition checks were met. 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting lvm2-monitor.service - Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting [email protected] - Load Kernel Module configfs... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting [email protected] - Load Kernel Module drm... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting [email protected] - Load Kernel Module fuse... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting systemd-journald.service - Journal Service... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: systemd-pcrmachine.service - TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionSecurity=measured-uki). 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: systemd-tpm2-setup-early.service - TPM2 SRK Setup (Early) was skipped because of an unmet condition check (ConditionSecurity=measured-uki). 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System. 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System. 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System. 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System. 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: [email protected]: Deactivated successfully. 9月 20 16:24:32 VM-0-6-tencentos systemd-journald[767]: Collecting audit messages is disabled. 9月 20 16:24:32 VM-0-6-tencentos kernel: fuse: init (API version 7.39) 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Finished [email protected] - Load Kernel Module configfs. 9月 20 16:24:32 VM-0-6-tencentos systemd[1]: Started systemd-journald.service - Journal Service. 9月 20 16:24:32 VM-0-6-tencentos kernel: ACPI: bus type drm_connector registered 9月 20 16:24:33 VM-0-6-tencentos systemd-journald[767]: Received client request to flush runtime journal. 9月 20 16:24:33 VM-0-6-tencentos systemd-journald[767]: /var/log/journal/2fa88b3fbc4d497eaa61f948bcf01cb9/system.journal: Journal file uses a different sequence number ID, rotating. 9月 20 16:24:33 VM-0-6-tencentos systemd-journald[767]: Rotating system journal. 9月 20 16:24:33 VM-0-6-tencentos kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input2 9月 20 16:24:33 VM-0-6-tencentos kernel: ACPI: button: Power Button [PWRF] 9月 20 16:24:33 VM-0-6-tencentos kernel: input: PC Speaker as /devices/platform/pcspkr/input/input3 9月 20 16:24:33 VM-0-6-tencentos kernel: piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 9月 20 16:24:33 VM-0-6-tencentos kernel: XFS (vdb): Mounting V5 Filesystem 2435a2e9-47a9-42c4-b1a9-3492b8ef3232 9月 20 16:24:33 VM-0-6-tencentos kernel: cirrus 0000:00:02.0: vgaarb: deactivate vga console 9月 20 16:24:33 VM-0-6-tencentos kernel: Console: switching to colour dummy device 80x25 9月 20 16:24:33 VM-0-6-tencentos kernel: [drm] Initialized cirrus 2.0.0 2019 for 0000:00:02.0 on minor 0 9月 20 16:24:33 VM-0-6-tencentos kernel: fbcon: cirrusdrmfb (fb0) is primary device 9月 20 16:24:33 VM-0-6-tencentos kernel: XFS (vdb): Ending clean mount 9月 20 16:24:33 VM-0-6-tencentos kernel: Console: switching to colour frame buffer device 128x48 9月 20 16:24:33 VM-0-6-tencentos kernel: cirrus 0000:00:02.0: [drm] fb0: cirrusdrmfb frame buffer device 9月 20 16:24:33 VM-0-6-tencentos kernel: RPC: Registered named UNIX socket transport module. 9月 20 16:24:33 VM-0-6-tencentos kernel: RPC: Registered udp transport module. 9月 20 16:24:33 VM-0-6-tencentos kernel: RPC: Registered tcp transport module. 9月 20 16:24:33 VM-0-6-tencentos kernel: RPC: Registered tcp-with-tls transport module. 9月 20 16:24:33 VM-0-6-tencentos kernel: RPC: Registered tcp NFSv4.1 backchannel transport module. 9月 20 16:24:34 VM-0-6-tencentos kernel: ISO 9660 Extensions: Microsoft Joliet Level 3 9月 20 16:24:34 VM-0-6-tencentos kernel: ISO 9660 Extensions: RRIP_1991A 9月 20 16:24:34 VM-0-6-tencentos kernel: ISO 9660 Extensions: Microsoft Joliet Level 3 9月 20 16:24:34 VM-0-6-tencentos kernel: ISO 9660 Extensions: RRIP_1991A 9月 20 16:24:34 VM-0-6-tencentos kernel: 8021q: adding VLAN 0 to HW filter on device eth0 9月 20 16:24:34 VM-0-6-tencentos kernel: IPMI message handler: version 39.2 9月 20 16:24:34 VM-0-6-tencentos kernel: ipmi device interface 9月 20 16:24:34 VM-0-6-tencentos kernel: ipmi_si: IPMI System Interface driver 9月 20 16:24:34 VM-0-6-tencentos kernel: ipmi_si: Unable to find any System Interface(s) 9月 20 16:24:35 VM-0-6-tencentos kernel: block vda: the capability attribute has been deprecated. 9月 20 16:24:35 VM-0-6-tencentos kernel: ISO 9660 Extensions: Microsoft Joliet Level 3 9月 20 16:24:35 VM-0-6-tencentos kernel: ISO 9660 Extensions: RRIP_1991A 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 1(veth6c05fad) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 1(veth6c05fad) entered disabled state 9月 20 16:24:37 VM-0-6-tencentos kernel: veth6c05fad: entered allmulticast mode 9月 20 16:24:37 VM-0-6-tencentos kernel: veth6c05fad: entered promiscuous mode 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 1(veth6c05fad) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 1(veth6c05fad) entered forwarding state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 1(veth6c05fad) entered disabled state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 2(vethef8fcbc) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 2(vethef8fcbc) entered disabled state 9月 20 16:24:37 VM-0-6-tencentos kernel: vethef8fcbc: entered allmulticast mode 9月 20 16:24:37 VM-0-6-tencentos kernel: vethef8fcbc: entered promiscuous mode 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 2(vethef8fcbc) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 2(vethef8fcbc) entered forwarding state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 2(vethef8fcbc) entered disabled state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 3(veth82f3ebc) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 3(veth82f3ebc) entered disabled state 9月 20 16:24:37 VM-0-6-tencentos kernel: veth82f3ebc: entered allmulticast mode 9月 20 16:24:37 VM-0-6-tencentos kernel: veth82f3ebc: entered promiscuous mode 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 3(veth82f3ebc) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 3(veth82f3ebc) entered forwarding state 9月 20 16:24:37 VM-0-6-tencentos kernel: eth0: renamed from vetha1cbfcc 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 3(veth82f3ebc) entered disabled state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 1(veth6c05fad) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 1(veth6c05fad) entered forwarding state 9月 20 16:24:37 VM-0-6-tencentos kernel: eth0: renamed from veth807343f 9月 20 16:24:37 VM-0-6-tencentos kernel: eth0: renamed from veth11f3145 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 2(vethef8fcbc) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 2(vethef8fcbc) entered forwarding state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 3(veth82f3ebc) entered blocking state 9月 20 16:24:37 VM-0-6-tencentos kernel: br-f323d309da84: port 3(veth82f3ebc) entered forwarding state 9月 20 16:24:43 VM-0-6-tencentos kernel: br-cd57826e4de1: port 1(veth61ccaa6) entered blocking state 9月 20 16:24:43 VM-0-6-tencentos kernel: br-cd57826e4de1: port 1(veth61ccaa6) entered disabled state 9月 20 16:24:43 VM-0-6-tencentos kernel: veth61ccaa6: entered allmulticast mode 9月 20 16:24:43 VM-0-6-tencentos kernel: veth61ccaa6: entered promiscuous mode 9月 20 16:24:43 VM-0-6-tencentos kernel: br-cd57826e4de1: port 1(veth61ccaa6) entered blocking state 9月 20 16:24:43 VM-0-6-tencentos kernel: br-cd57826e4de1: port 1(veth61ccaa6) entered forwarding state 9月 20 16:24:43 VM-0-6-tencentos kernel: br-cd57826e4de1: port 1(veth61ccaa6) entered disabled state 9月 20 16:24:43 VM-0-6-tencentos kernel: eth0: renamed from vetha4c7914 9月 20 16:24:43 VM-0-6-tencentos kernel: br-cd57826e4de1: port 1(veth61ccaa6) entered blocking state 9月 20 16:24:43 VM-0-6-tencentos kernel: br-cd57826e4de1: port 1(veth61ccaa6) entered forwarding state 9月 20 16:24:49 VM-0-6-tencentos kernel: FS-Cache: Loaded
根据补充的信息，可以基本排除 宿主机 OOM 和 systemd Memory 限制
当前结论调整：
cubelet 进程不是因为内存不足被杀，而是 cubelet 自己的启动流程里主动退出/被父脚本处理，或者触发了某个运行时检查
关键证据
1. 宿主机资源完全正常
你的机器：
Mem: 125Gi
used: 9.7Gi
available:115Gi
Swap:0
内存非常充足
同时：
dmesg | grep -Ei "oom|killed"
无输出
排除：
- ❌ kernel OOM killer
- ❌ docker OOM
- ❌ 内存不足
2. 当前 cubelet 实际运行正常
现在：
ps aux | grep cubelet
看到：
root 1949
/usr/local/services/cubetoolbox/Cubelet/bin/cubelet
状态：
Sl
说明：
- S = sleeping
- l = multi-thread
进程正常
3. 真正异常是这个：
启动：
cubelet initial pid=1710
马上：
1710 Killed
然后：
cubelet stable pid=1949
也就是说：
启动脚本：
cubelet-start.sh
逻辑类似：
start cubelet &
wait
if alive
use it
else
restart
fi
第一次 pid 1710 被杀，然后重新拉起 1949
所以不是服务失败
下一步重点查 cubelet-start.sh
执行：
sed -n '1,150p' \
/usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh
重点看：
第57行附近
你的日志：
cubelet-start.sh: line 57:
1710 Killed
需要看：
nl -ba \
/usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh \
| sed -n '45,70p'
第二个重点：为什么第一次启动被杀？
目前可能性：
可能1：cubelet 自带 watchdog/reexec
很多云沙箱组件启动：
bootstrap process
|
|
+--> real cubelet
第一次 pid：
1710
只是 bootstrap
然后：
exec/reload
换 pid：
1949
脚本误判为 killed
可能2：动态配置 reload
日志：
listen file[
/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
]
说明 cubelet 有动态配置监听
启动流程：
cubelet
|
加载config
|
读取dynamicconf
|
重新fork
|
旧pid退出
|
新pid接管
也可能显示：
Killed
但实际是正常行为
3. 检查 cubelet 健康接口
先确认：
ss -lntp | grep cubelet
或者：
netstat -lntp | grep 1949
看监听端口
4. 查看 cubelet 日志目录
找日志：
find /usr/local/services/cubetoolbox/Cubelet \
-name "*.log" \
-o -name "*.out"
重点：
ls -lh /usr/local/services/cubetoolbox/Cubelet/log/
5. 检查是否有 sandbox runtime
你的 docker：
没有看到：
qemu
builder
template
只有：
cube-proxy
cube-egress
cube-webui
cube-lifecycle-manager
redis
mysql
minio
coredns
说明：
目前只是 control plane
还没有创建 sandbox
执行：
docker ps -a | grep -Ei "qemu|sandbox|builder|template"
如果为空：
正常
6. lifecycle-manager 状态正常
这一块不用处理：
bootstrap complete entries=0
proxy joined
replay done
说明：
cube-master
cube-proxy
lifecycle-manager
redis
通信正常
我建议现在执行这几个命令
按顺序：
① 看启动脚本第57行
nl -ba \
/usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh \
| sed -n '45,70p'
② 看 cubelet 日志
find /usr/local/services/cubetoolbox/Cubelet \
-type f | grep -Ei "log|err"
③ 看端口
ss -lntp | grep cube
④ 看 cubelet完整启动参数
cat /proc/1949/cmdline | tr '\0' ' '
目前判断：
| 项目 | 状态 |
|---|---|
| 宿主机内存 | ✅正常 |
| 磁盘 | ✅正常 |
| Docker组件 | ✅正常 |
| Lifecycle Manager | ✅正常 |
| Cubelet最终进程 | ✅正常 |
| 首次pid killed | ⚠️启动脚本/自重启机制 |
下一步主要确认 cubelet-start.sh 第57行到底是什么逻辑
nl -ba \ /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh \ | sed -n '45,70p' 45 46 if [[ -n "${CUBE_SANDBOX_NODE_IP:-}" ]]; then 47 ( 48 export CUBE_SANDBOX_NODE_IP="${CUBE_SANDBOX_NODE_IP}" 49 exec "${CUBELET_BIN}" --config "${CUBELET_CONFIG}" --dynamic-conf-path "${CUBELET_DYNAMICCONF}" 50 ) & 51 else 52 "${CUBELET_BIN}" --config "${CUBELET_CONFIG}" --dynamic-conf-path "${CUBELET_DYNAMICCONF}" & 53 fi 54 initial_pid=$! 55 log "cubelet initial pid=${initial_pid}" 56 57 for _ in {1..20}; do 58 if ! kill -0 "${initial_pid}" >/dev/null 2>&1; then 59 break 60 fi 61 sleep 1 62 done 63 64 refresh_pidfile_from_pattern "${PID_FILE}" "${PATTERN}" 20 1 || die "failed to determine cubelet child pid" 65 log "cubelet stable pid=$(<"${PID_FILE}")" [root@VM-0-6-tencentos ~]# find /usr/local/services/cubetoolbox/Cubelet \ -type f | grep -Ei "log|err" [root@VM-0-6-tencentos ~]# ss -lntp | grep cube LISTEN 0 4096 0.0.0.0:3010 0.0.0.0:* users:(("cubeops",pid=4673,fd=10)) LISTEN 0 4096 0.0.0.0:3000 0.0.0.0:* users:(("cube-api",pid=5903,fd=9)) LISTEN 0 4096 0.0.0.0:8083 0.0.0.0:* users:(("cube-lifecycle-",pid=5708,fd=9)) LISTEN 0 4096 0.0.0.0:8089 0.0.0.0:* users:(("cubemaster",pid=5538,fd=15)) LISTEN 0 4096 0.0.0.0:9999 0.0.0.0:* users:(("cubelet",pid=1949,fd=636)) LISTEN 0 4096 0.0.0.0:9998 0.0.0.0:* users:(("cubelet",pid=1949,fd=633)) LISTEN 0 4096 0.0.0.0:9966 0.0.0.0:* users:(("cubelet",pid=1949,fd=631)) [root@VM-0-6-tencentos ~]# cat /proc/1949/cmdline | tr '\0' ' ' /usr/local/services/cubetoolbox/Cubelet/bin/cubelet --config /usr/local/services/cubetoolbox/Cubelet/config/config.toml --dynamic-conf-path /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml
从目前信息看，cubelet 并没有真正异常退出，之前日志里的：
cubelet-start.sh: line 57: 6023 Killed
cubelet-start.sh: line 57: 1668 Killed
cubelet-start.sh: line 57: 1710 Killed
不是 cubelet 服务最终 PID 被杀，而是 启动脚本 fork 出来的初始子进程被替换/退出。现在稳定运行：
root 1949 0.5 0.1 4164304 221340 ? Sl 16:24 0:31
/usr/local/services/cubetoolbox/Cubelet/bin/cubelet
并且监听：
0.0.0.0:9999 cubelet
0.0.0.0:9998 cubelet
0.0.0.0:9966 cubelet
说明 cubelet 已正常启动
1. 先解释为什么看到 Killed
你的脚本：
if [[ -n "${CUBE_SANDBOX_NODE_IP:-}" ]]; then
(
export CUBE_SANDBOX_NODE_IP="${CUBE_SANDBOX_NODE_IP}"
exec "${CUBELET_BIN}" ...
) &
fi
initial_pid=$!
这里：
6023
1668
1710
是 shell fork 出来的 wrapper pid
启动后：
exec cubelet
替换进程。
然后脚本继续：
refresh_pidfile_from_pattern
重新通过：
PATTERN
找到真正 cubelet PID
所以：
Killed 1710
stable pid=1949
实际过程：
cubelet-start.sh
|
+-- bash subshell PID 1710
|
exec cubelet
|
PID变成cubelet
最终：
PID 1949 cubelet
这个行为类似 Kubernetes kubelet wrapper
2. 但是有一个值得检查的问题
你的日志：
第一次：
cubelet initial pid=6023
Killed 6023
cubelet stable pid=7079
第二次：
cubelet initial pid=1668
Killed 1668
stable pid=2125
第三次：
initial pid=1710
Killed 1710
stable pid=1949
每次都是：
启动后约 1 秒：
Killed
说明 wrapper 检测机制可能有问题
建议检查：
cat /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh
前面定义：
PATTERN=
重点看：
refresh_pidfile_from_pattern
匹配规则
执行：
grep PATTERN -n \
/usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh
3. 当前 cubelet 状态确认
执行：
systemctl status cube-sandbox-cubelet
应该类似：
Active: active (running)
Main PID: 1949
再看：
cat /run/cube-sandbox-systemd/cubelet.pid
应该：
1949
确认 systemd 管理的是正确 PID
4. 当前资源没有问题
你的机器：
Mem:
125Gi total
9.7Gi used
115Gi available
Swap:
0
不存在 OOM
同时：
dmesg -T | grep -Ei "oom|killed"
为空
所以排除：
- 内存不足
- kernel OOM killer
- cgroup memory kill
5. cubelet 自己日志在哪里？
你找：
find /usr/local/services/cubetoolbox/Cubelet \
-type f | grep -Ei "log|err"
没有输出
说明它大概率写 stdout
systemd 里查看：
journalctl \
-u cube-sandbox-cubelet \
-f
然后：
另开窗口：
curl http://127.0.0.1:9999
或者：
curl http://127.0.0.1:9966
观察
6. 建议检查 cubelet health
你的端口：
| 端口 | 可能用途 |
|---|---|
| 9966 | metrics |
| 9998 | RPC |
| 9999 | HTTP API |
测试：
curl -v http://127.0.0.1:9999/health
或者：
curl -v http://127.0.0.1:9999/healthz
查看：
curl -s http://127.0.0.1:9966/metrics | head
如果返回 Prometheus 指标：
说明完全正常
7. 我认为当前真正问题点
从你的日志：
cube-lifecycle-manager
bootstrap complete
entries:0
proxy joined
replay done pushed:0 failed:0
控制面正常
Docker:
cube-proxy
cube-webui
cube-egress
redis
mysql
minio
全部 healthy
cubelet:
PID 1949
ports opened
正常
目前唯一异常：
cubelet-start.sh wrapper pid killed
但不影响服务
cat /run/cube-sandbox-systemd/cubelet.pid systemctl status cube-sandbox-cubelet grep PATTERN -n \ /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh curl -s http://127.0.0.1:9966/metrics | head -20 curl -v http://127.0.0.1:9999/health 1949 ● cube-sandbox-cubelet.service - Cube Sandbox Cubelet Loaded: loaded (/etc/systemd/system/cube-sandbox-cubelet.service; disabled; preset: disabled) Active: active (running) since Sun 2026-09-20 16:24:36 CST; 1h 46min ago Process: 1505 ExecStartPre=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/prepare-compute-role.sh (code=exited, status=0/SUCCESS) Process: 1580 ExecStart=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh (code=exited, status=0/SUCCESS) Main PID: 1949 (cubelet) Tasks: 0 (limit: 153562) Memory: 212.4M (peak: 231.0M) CPU: 10.815s CGroup: /system.slice/cube-sandbox-cubelet.service ‣ 1949 /usr/local/services/cubetoolbox/Cubelet/bin/cubelet --config /usr/local/services/cubetoolbox/Cubelet/config/config.toml --dynamic-conf-path /usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml 9月 20 16:24:35 VM-0-6-tencentos bash[1608]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 16:24:35 VM-0-6-tencentos bash[1580]: [one-click-systemd] updated cubelet cow.s3 enable=false 9月 20 16:24:35 VM-0-6-tencentos bash[1580]: [one-click-systemd] cubelet initial pid=1710 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026-09-20T16:24:35.552388365+08:00,listen file[/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml] 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: cfg:{"Common":{"CommonTimeout":10000000000,"LogLevel":"","DescribeAsyncInterval":100000000,"EnablePFMode":false,"DescribeBDFInterval":10000000,"DescribeBDFTimeout":10000000000,"GetBDFByUuidCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_uuid","GetBDFByIfNameCmd":"/usr/local/services> 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026/09/20 16:24:35 mounted state tmpfs /data/cubelet/state size=1024m 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026/09/20 16:24:35 set /data/cubelet/root as shared mount 9月 20 16:24:36 VM-0-6-tencentos bash[1580]: /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh: line 57: 1710 Killed ( export CUBE_SANDBOX_NODE_IP="${CUBE_SANDBOX_NODE_IP}"; exec "${CUBELET_BIN}" --config "${CUBELET_CONFIG}" --dynamic-conf-path "${CUBELET_DYNAMICCONF}" ) 9月 20 16:24:36 VM-0-6-tencentos bash[1580]: [one-click-systemd] cubelet stable pid=1949 9月 20 16:24:36 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cubelet.service - Cube Sandbox Cubelet. 19:PATTERN="^${CUBELET_BIN} --config" 39: if [[ -n "${existing_pid}" ]] && kill -0 "${existing_pid}" >/dev/null 2>&1 && pid_matches_pattern "${existing_pid}" "${PATTERN}"; then 64:refresh_pidfile_from_pattern "${PID_FILE}" "${PATTERN}" 20 1 || die "failed to determine cubelet child pid" 404 page not found * Trying 127.0.0.1:9999... * Connected to 127.0.0.1 (127.0.0.1) port 9999 > GET /health HTTP/1.1 > Host: 127.0.0.1:9999 > User-Agent: curl/8.4.0 > Accept: */* > * Received HTTP/0.9 when not allowed * Closing connection curl: (1) Received HTTP/0.9 when not allowed [root@VM-0-6-tencentos ~]# journalctl -u cube-sandbox-cubelet -f docker logs cube-proxy docker logs cube-lifecycle-manager 9月 20 16:24:35 VM-0-6-tencentos bash[1608]: [one-click-systemd] updated cubelet cubeops_addr=http://127.0.0.1:3010 9月 20 16:24:35 VM-0-6-tencentos bash[1580]: [one-click-systemd] updated cubelet cow.s3 enable=false 9月 20 16:24:35 VM-0-6-tencentos bash[1580]: [one-click-systemd] cubelet initial pid=1710 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026-09-20T16:24:35.552388365+08:00,listen file[/usr/local/services/cubetoolbox/Cubelet/dynamicconf/conf.yaml] 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: cfg:{"Common":{"CommonTimeout":10000000000,"LogLevel":"","DescribeAsyncInterval":100000000,"EnablePFMode":false,"DescribeBDFInterval":10000000,"DescribeBDFTimeout":10000000000,"GetBDFByUuidCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_uuid","GetBDFByIfNameCmd":"/usr/local/services/AdamPlugins-1.0/snhost_snic_sdk/cath/bm/tools/get_bdf_by_ifname","CommandTimeout":1000000000,"DisableHostCgroup":true,"DisableVmCgroup":false,"EnableSandboxExecCmdBeforeExist":false,"SandboxExecCmdTimeOut":5000000000,"SandboxExecCmdBeforeExist":["sh","/usr/local/cubetools/cube_exec_cmd_before_exist.sh"],"SandboxExecCmdBeforeExistLogOut":false,"SandboxExecCmdOutMaxLines":2000,"SandboxExecCmdMatchLine":"","SandboxExecCmdAfterMatch":null,"SandboxExecCmdBase64AfterMatch":"","SandboxExecCmdAfterMatchLogOut":false,"CgroupDisableMemoryReparentFile":"","CgroupDisableCpusetList":"","DisableHostNetfile":true,"DefaultDNSServers":["119.29.29.29"],"DefaultDNSSearches":[],"DefaultDNSOptions":[],"ReconcileInterval":300000000000,"DisableCubeBoxTemplateBaseFormatPoolOfNumberVer":false},"Tenant":{"Tenants":{},"UseRespectiveNamespace":false,"DefaultNS":"default","ZiyanUinListStr":""},"MetaServerConfig":{"MetaServerEndpoint":"127.0.0.1:3010","NodeStatusMaxImages":40000,"CubeMasterHTTPAddr":"127.0.0.1:8089"},"HostConf":{"Quota":{"Cpu":0,"Mem":"","MvmLimit":0,"CreationConcurrentNum":0,"PausedResourceReleaseRatio":0},"GC":{"CodeExpirationTime":"72h","ImageExpirationTime":"24h"},"SchedulerLabel":"default-cluster"}} 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026/09/20 16:24:35 mounted state tmpfs /data/cubelet/state size=1024m 9月 20 16:24:35 VM-0-6-tencentos bash[1949]: 2026/09/20 16:24:35 set /data/cubelet/root as shared mount 9月 20 16:24:36 VM-0-6-tencentos bash[1580]: /usr/local/services/cubetoolbox/scripts/systemd/cubelet-start.sh: line 57: 1710 Killed ( export CUBE_SANDBOX_NODE_IP="${CUBE_SANDBOX_NODE_IP}"; exec "${CUBELET_BIN}" --config "${CUBELET_CONFIG}" --dynamic-conf-path "${CUBELET_DYNAMICCONF}" ) 9月 20 16:24:36 VM-0-6-tencentos bash[1580]: [one-click-systemd] cubelet stable pid=1949 9月 20 16:24:36 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cubelet.service - Cube Sandbox Cubelet. ^C {"level":"info","ts":1789892682.4259038,"caller":"cube-lifecycle-manager/main.go:59","msg":"cube-lifecycle-manager starting","redis_addr":"127.0.0.1:6379","cube_proxy_admin_urls":["http://127.0.0.1:8082"],"cubemaster_url":"http://192.168.0.6:8089","listen_addr":"0.0.0.0:8083","instance_identity":"VM-0-6-tencentos","leader_election_enabled":false} {"level":"info","ts":1789892682.428625,"caller":"cube-lifecycle-manager/main.go:81","msg":"eventbus enabled"} {"level":"info","ts":1789892682.4323788,"caller":"cube-lifecycle-manager/main.go:641","msg":"bootstrap complete","entries":0} {"level":"info","ts":1789892682.4353082,"logger":"http","caller":"httpapi/server.go:86","msg":"cube-lifecycle-manager http server listening","addr":"0.0.0.0:8083"} {"level":"info","ts":1789892685.4343956,"logger":"discovery","caller":"discovery/redis.go:182","msg":"discovery: proxy joined","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082"} {"level":"info","ts":1789892685.434483,"logger":"replay","caller":"cube-lifecycle-manager/main.go:366","msg":"replay begin","proxy_id":"192.168.0.6:8082","admin_url":"http://192.168.0.6:8082","entries":0} {"level":"info","ts":1789892685.434507,"logger":"replay","caller":"cube-lifecycle-manager/main.go:394","msg":"replay done","proxy_id":"192.168.0.6:8082","pushed":0,"failed":0} [root@VM-0-6-tencentos ~]# ls -lah /data/cubelet ls -lah /data/cube-shim ls -lah /data/snapshot_pack 总计 4.0K drwxr-x--- 6 root root 127 9月20日 16:24 . drwxr-xr-x 10 root root 136 9月20日 15:36 .. drwxr-xr-x 3 root root 16 9月20日 15:36 cleanup srw-rw---- 1 root root 0 9月20日 16:24 cubelet.sock srw-rw---- 1 root root 0 9月20日 16:24 cubelet.sock.ttrpc srw-rw---- 1 root root 0 9月20日 16:24 cubetap.sock drwxr-x--- 13 root root 4.0K 9月20日 15:36 root drwx--x--x 2 root root 6 9月20日 15:36 state drwxr-xr-x 4 root root 55 9月20日 15:36 storage 总计 0 drwxr-x--- 3 root root 19 9月20日 15:36 . drwxr-xr-x 10 root root 136 9月20日 15:36 .. drwxr-x--- 2 root root 6 9月20日 15:36 disks 总计 0 drwxr-x--- 3 root root 19 9月20日 15:36 . drwxr-xr-x 10 root root 136 9月20日 15:36 .. drwxr-x--- 2 root root 6 9月20日 15:36 disks
根据补充的结果，Cube Sandbox v0.7.1 控制面和 cubelet 本身已经正常运行。之前怀疑的 OOM、磁盘、systemd PID 管理问题基本可以排除
下面是详细判断
1. cubelet PID 管理正常
现在：
cat /run/cube-sandbox-systemd/cubelet.pid
结果：
1949
systemd：
Main PID: 1949 (cubelet)
完全一致
说明：
cubelet-start.sh
|
+-- 临时启动PID 1710
|
+-- exec cubelet
|
+-- 最终PID 1949
|
+-- 写入pidfile
这个流程正常
2. Killed 不是异常
关键：
cubelet-start.sh: line 57: 1710 Killed
cubelet stable pid=1949
这里的 1710：
(
export CUBE_SANDBOX_NODE_IP
exec cubelet
) &
产生的 subshell PID
后续：
refresh_pidfile_from_pattern
找到：
/usr/local/services/cubetoolbox/Cubelet/bin/cubelet --config
真正 PID：
1949
所以不用处理
3. PATTERN 匹配也正确
你的：
19:PATTERN="^${CUBELET_BIN} --config"
当前：
cat /proc/1949/cmdline | tr '\0' ' '
输出：
/usr/local/services/cubetoolbox/Cubelet/bin/cubelet --config ...
匹配：
^cubelet --config
OK
4. cubelet API 判断
你执行：
curl -s http://127.0.0.1:9966/metrics
返回：
404 page not found
说明：
9966 有 HTTP server
只是：
/metrics
不是暴露路径
不是故障
9999：
Received HTTP/0.9 when not allowed
这个非常关键
说明：
9999 不是普通 HTTP/1.1 REST
可能是：
- gRPC
- ttrpc
- 自定义 TCP protocol
所以：
curl http://127.0.0.1:9999/health
方式不适用
5. 当前目录状态正常
你的：
/data/cubelet
里面：
cubelet.sock
cubelet.sock.ttrpc
cubetap.sock
root/
state/
storage/
说明：
cubelet runtime 初始化成功
特别：
cubelet.sock.ttrpc
说明它已经进入 sandbox runtime 阶段
现在真正需要排查的是：为什么没有 Sandbox / Template
因为：
控制面：
cubeops 3010
cube-api 3000
cubemaster 8089
lifecycle 8083
proxy 8082
全部起来
但是：
/data/snapshot_pack/disks
为空：
drwxr-x--- 2 root root 6 disks
说明：
目前还没有：
- template
- snapshot
- image
- qemu disk
下一步重点检查
1. 看 cubelet 注册状态
执行：
curl -s http://127.0.0.1:3010/api/v1/nodes | jq
如果不知道 API：
先：
ss -lntp | grep 3010
确认 cubeops
2. 查看 cubelet RPC状态
找端口：
ss -lntp | grep cubelet
已有：
9966
9998
9999
尝试：
curl -v http://127.0.0.1:9966/
和：
curl -v http://127.0.0.1:9998/
看看返回
3. 检查 qemu / shim
你的：
/data/cube-shim/disks
为空
检查：
ls -lah /data/cube-shim
以及：
find /usr/local/services/cubetoolbox \
-type f \
-name "*qemu*"
4. 看 cube-proxy 日志
你只看了 lifecycle
执行：
docker logs --tail=200 cube-proxy
重点找：
error
failed
template
sandbox
create
node
cubelet
5. 看 cubemaster
目前没有看
执行：
journalctl -u cube-sandbox-cubemaster -n 200 --no-pager
或者：
ps aux | grep cubemaster
6. 检查模板目录
执行：
find /data -maxdepth 3 \
-type d \
| grep -Ei "template|image|snapshot|box"
正常安装后通常应该有类似：
template/
snapshot/
image/
rootfs/
当前状态总结
| 组件 | 状态 |
|---|---|
| cube-api | ✅ |
| cubeops | ✅ |
| cubemaster | ✅ |
| cube-proxy | ✅ |
| cube-lifecycle-manager | ✅ |
| redis | ✅ |
| mysql | ✅ |
| minio | ✅ |
| cubelet | ✅ |
| OOM | ❌ 排除 |
| 磁盘压力 | ❌ 排除 |
| cubelet启动失败 | ❌ 排除 |
| template/image | ⚠️ 未发现 |
[root@VM-0-6-tencentos ~]# docker logs --tail=200 cube-proxy [root@VM-0-6-tencentos ~]# ps aux | grep cubemaster root 5538 1.6 0.0 3566552 42072 ? Ssl 16:24 2:01 /usr/local/services/cubetoolbox/CubeMaster/bin/cubemaster root 214146 0.0 0.0 6696 2060 pts/0 S+ 18:24 0:00 grep --color=auto cubemaster [root@VM-0-6-tencentos ~]# journalctl -u cube-sandbox-cubemaster -n 200 --no-pager 9月 20 17:54:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:54:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:54:12 VM-0-6-tencentos bash[5538]: [0.332ms] [rows:0] SELECT * FROM
t_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:54:12 VM-0-6-tencentos bash[5538]: 9月 20 17:54:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:54:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:54:12 VM-0-6-tencentos bash[5538]: [0.502ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:54:42 VM-0-6-tencentos bash[5538]: 9月 20 17:54:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:54:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:54:42 VM-0-6-tencentos bash[5538]: [0.362ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:54:42 VM-0-6-tencentos bash[5538]: 9月 20 17:54:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:54:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:54:42 VM-0-6-tencentos bash[5538]: [0.419ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:55:12 VM-0-6-tencentos bash[5538]: 9月 20 17:55:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:55:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:55:12 VM-0-6-tencentos bash[5538]: [0.421ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:55:12 VM-0-6-tencentos bash[5538]: 9月 20 17:55:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:55:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:55:12 VM-0-6-tencentos bash[5538]: [0.374ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:55:42 VM-0-6-tencentos bash[5538]: 9月 20 17:55:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:55:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:55:42 VM-0-6-tencentos bash[5538]: [0.376ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:55:42 VM-0-6-tencentos bash[5538]: 9月 20 17:55:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:55:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:55:42 VM-0-6-tencentos bash[5538]: [0.395ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:56:12 VM-0-6-tencentos bash[5538]: 9月 20 17:56:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:56:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:56:12 VM-0-6-tencentos bash[5538]: [0.358ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:56:12 VM-0-6-tencentos bash[5538]: 9月 20 17:56:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:56:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:56:12 VM-0-6-tencentos bash[5538]: [0.401ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:56:42 VM-0-6-tencentos bash[5538]: 9月 20 17:56:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:56:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:56:42 VM-0-6-tencentos bash[5538]: [0.384ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:56:42 VM-0-6-tencentos bash[5538]: 9月 20 17:56:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:56:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:56:42 VM-0-6-tencentos bash[5538]: [0.406ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:57:12 VM-0-6-tencentos bash[5538]: 9月 20 17:57:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:57:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:57:12 VM-0-6-tencentos bash[5538]: [0.418ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:57:12 VM-0-6-tencentos bash[5538]: 9月 20 17:57:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:57:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:57:12 VM-0-6-tencentos bash[5538]: [0.376ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:57:42 VM-0-6-tencentos bash[5538]: 9月 20 17:57:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:57:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:57:42 VM-0-6-tencentos bash[5538]: [0.346ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:57:42 VM-0-6-tencentos bash[5538]: 9月 20 17:57:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:57:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:57:42 VM-0-6-tencentos bash[5538]: [0.351ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:58:12 VM-0-6-tencentos bash[5538]: 9月 20 17:58:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:58:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:58:12 VM-0-6-tencentos bash[5538]: [0.360ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:58:12 VM-0-6-tencentos bash[5538]: 9月 20 17:58:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:58:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:58:12 VM-0-6-tencentos bash[5538]: [0.352ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:58:42 VM-0-6-tencentos bash[5538]: 9月 20 17:58:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:58:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:58:42 VM-0-6-tencentos bash[5538]: [0.379ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:58:42 VM-0-6-tencentos bash[5538]: 9月 20 17:58:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:58:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:58:42 VM-0-6-tencentos bash[5538]: [0.342ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:59:12 VM-0-6-tencentos bash[5538]: 9月 20 17:59:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:59:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:59:12 VM-0-6-tencentos bash[5538]: [0.332ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:59:12 VM-0-6-tencentos bash[5538]: 9月 20 17:59:12 VM-0-6-tencentos bash[5538]: 2026/09/20 17:59:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:59:12 VM-0-6-tencentos bash[5538]: [0.341ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 17:59:42 VM-0-6-tencentos bash[5538]: 9月 20 17:59:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:59:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 17:59:42 VM-0-6-tencentos bash[5538]: [0.394ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 17:59:42 VM-0-6-tencentos bash[5538]: 9月 20 17:59:42 VM-0-6-tencentos bash[5538]: 2026/09/20 17:59:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 17:59:42 VM-0-6-tencentos bash[5538]: [0.366ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 18:00:12 VM-0-6-tencentos bash[5538]: 9月 20 18:00:12 VM-0-6-tencentos bash[5538]: 2026/09/20 18:00:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 18:00:12 VM-0-6-tencentos bash[5538]: [0.400ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 18:00:12 VM-0-6-tencentos bash[5538]: 9月 20 18:00:12 VM-0-6-tencentos bash[5538]: 2026/09/20 18:00:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 18:00:12 VM-0-6-tencentos bash[5538]: [0.385ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 18:00:42 VM-0-6-tencentos bash[5538]: 9月 20 18:00:42 VM-0-6-tencentos bash[5538]: 2026/09/20 18:00:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 18:00:42 VM-0-6-tencentos bash[5538]: [0.382ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 18:00:42 VM-0-6-tencentos bash[5538]: 9月 20 18:00:42 VM-0-6-tencentos bash[5538]: 2026/09/20 18:00:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 18:00:42 VM-0-6-tencentos bash[5538]: [0.407ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 18:01:12 VM-0-6-tencentos bash[5538]: 9月 20 18:01:12 VM-0-6-tencentos bash[5538]: 2026/09/20 18:01:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 18:01:12 VM-0-6-tencentos bash[5538]: [0.360ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 18:01:12 VM-0-6-tencentos bash[5538]: 9月 20 18:01:12 VM-0-6-tencentos bash[5538]: 2026/09/20 18:01:12 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 18:01:12 VM-0-6-tencentos bash[5538]: [0.417ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 18:01:42 VM-0-6-tencentos bash[5538]: 9月 20 18:01:42 VM-0-6-tencentos bash[5538]: 2026/09/20 18:01:42 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 18:01:42 VM-0-6-tencentos bash[5538]: [0.389ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1 9月 20 18:01:42 VM-0-6-tencentos bash[5538]: 9月 20 18:01:42 VM-0-6-tencentos bash[5538]: 2026/09/20 18:01:42 /workspace/CubeMaster/pkg/templatecenter/snapshot_table.go:29 record not found 9月 20 18:01:42 VM-0-6-tencentos bash[5538]: [0.374ms] [rows:0] SELECT * FROMt_cube_snapshotWHERE snapshot_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_snapshot.deleted_atIS NULL ORDER BYt_cube_snapshot.idLIMIT 1 9月 20 18:02:12 VM-0-6-tencentos bash[5538]: 9月 20 18:02:12 VM-0-6-tencentos bash[5538]: 2026/09/20 18:02:12 /workspace/CubeMaster/pkg/templatecenter/store.go:1118 record not found 9月 20 18:02:12 VM-0-6-tencentos bash[5538]: [0.389ms] [rows:0] SELECT * FROMt_cube_template_definitionWHERE template_id = 'tpl-464f1b5538764bd7afa240ca' ANDt_cube_template_definition.deleted_atIS NULL ORDER BYt_cube_template_definition.idLIMIT 1
从目前日志看，Cube Sandbox 主链路已经基本起来了：
- ✅
cubelet已正常运行 - ✅
cubemaster已正常运行 - ✅
cube-lifecycle-manager已正常运行 - ✅
cubeops :3010 - ✅
cube-api :3000 - ✅
cube-proxy可能无日志（需要确认） - ❌ 当前异常集中在 Template / Snapshot 数据缺失
重点不是 cubelet 启动问题，而是 模板中心状态不一致
1. cubelet 的 Killed 不是故障
这里：
cubelet-start.sh: line 57:
1710 Killed
实际发生：
initial pid=1710
...
stable pid=1949
你的脚本逻辑：
(
export CUBE_SANDBOX_NODE_IP
exec cubelet ...
) &
先启动一个探测进程：
1710
然后：
refresh_pidfile_from_pattern
找到真正 daemon：
1949
所以：
systemctl status
Main PID: 1949
正常
2. cubelet API 端口判断
你测试：
curl http://127.0.0.1:9999/health
返回：
HTTP/0.9
说明：
9999 不是 HTTP 服务
从监听：
9966
9998
9999
看：
大概率：
| 端口 | 用途 |
|---|---|
| 9966 | metrics |
| 9998 | ttrpc |
| 9999 | cubelet grpc/http2 |
所以：
curl 9999
不适用
测试：
curl http://127.0.0.1:9966/metrics | head
返回：
404 page not found
说明：
9966 也不是标准 prometheus path
建议查看：
cat /proc/1949/net/tcp
或者：
strings /usr/local/services/cubetoolbox/Cubelet/bin/cubelet | grep -Ei "metric|health|debug"
3. 当前真正问题：Template 不存在
核心日志：
SELECT * FROM t_cube_template_definition
WHERE template_id='tpl-464f1b5538764bd7afa240ca'
结果：
rows:0
还有：
SELECT * FROM t_cube_snapshot
WHERE snapshot_id='tpl-464f1b5538764bd7afa240ca'
结果：
rows:0
说明：
CubeMaster 收到请求：
template_id
tpl-464f1b5538764bd7afa240ca
但是数据库：
t_cube_template_definition
没有记录
4. 检查 TemplateCenter
你有：
/usr/local/services/cubetoolbox/CubeTemplateCenter
执行：
systemctl status cube-sandbox-cube-templatecenter
然后：
journalctl \
-u cube-sandbox-cube-templatecenter \
-n 200 \
--no-pager
重点看：
有没有：
mysql connect failed
redis failed
register template failed
5. 查数据库里的模板
需要找到 CubeMaster 数据库
先找配置：
grep -R "mysql\|database\|dsn" \
/usr/local/services/cubetoolbox \
-n
找到 DB：
例如：
mysql://user:pass@tcp(...)
进入 mysql：
mysql -h xxx -u xxx -p
查看：
show tables;
重点：
select count(*)
from t_cube_template_definition;
查看模板：
select
template_id,
name,
status
from t_cube_template_definition;
6. 检查默认镜像
你有：
/usr/local/services/cubetoolbox/cube-image/
里面：
cube-guest-image-cpu.img
version
但是：
/data/snapshot_pack
为空：
drwxr-xr-x snapshot_pack
└── disks
没有磁盘
说明：
目前只是安装了组件：
Cubelet
CubeMaster
TemplateCenter
但是：
模板镜像没有导入
7. 导入模板流程需要确认
搜索脚本：
find /usr/local/services/cubetoolbox \
-type f \
| grep -Ei "init|template|image|import|snapshot"
重点看：
cubetemplatecenter-postcheck.sh
内容：
cat \
/usr/local/services/cubetoolbox/scripts/systemd/cubetemplatecenter-postcheck.sh
8. 检查 TemplateCenter API
看看监听：
ss -lntp | grep template
或者：
docker ps
因为：
你现在：
CubeTemplateCenter/bin/templatecenter
应该是 systemd
查看：
systemctl status cube-sandbox-cube-templatecenter
9. 检查 cube-api 调用链
现在链路：
cube-api
|
cube-master :8089
|
TemplateCenter
|
DB
|
image storage
你的状态：
cube-api OK
cubemaster OK
cubelet OK
TemplateCenter 未确认
DB 有访问
Template数据 缺失
image 缺失
10. 下一步建议执行这组命令
请贴输出：
systemctl status cube-sandbox-cube-templatecenter
journalctl \
-u cube-sandbox-cube-templatecenter \
-n 100 \
--no-pager
grep -R "t_cube_template_definition" \
/usr/local/services/cubetoolbox \
-n
以及：
ls -lh \
/usr/local/services/cubetoolbox/cube-image/
目前判断：
不是 Cubelet、不是 systemd、不是网络问题
根因概率最高：
Cube Sandbox 安装完成，但默认 Guest Template / Snapshot 没有初始化导入，导致 CubeMaster 周期性查询不存在的 tpl-* 资源
[root@VM-0-6-tencentos ~]# systemctl status cube-sandbox-cube-templatecenter ● cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter Loaded: loaded (/etc/systemd/system/cube-sandbox-cube-templatecenter.service; enabled; preset: disabled) Active: active (running) since Sun 2026-09-20 16:24:44 CST; 2h 1min ago Process: 5910 ExecStartPost=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubetemplatecenter-postcheck.sh (code=exited, status=0/SUCCESS) Main PID: 5907 (templatecenter) Tasks: 28 (limit: 153562) Memory: 45.8M (peak: 48.5M) CPU: 9.938s CGroup: /system.slice/cube-sandbox-cube-templatecenter.service └─5907 /usr/local/services/cubetoolbox/CubeTemplateCenter/bin/templatecenter 9月 20 16:24:43 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter... 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: 2026-09-20T16:24:43.375476072+08:00,listen file[/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml] 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: cfg:{"Common":{"MockUpdateAction":false,"DebugDumpHttpBody":false,"MockDebug":false,"MockNodeNum":0,"MockCreateDirect":false,"MockCreateDirectHandle":false,"MockHttpDirect":false,"MockCreateSleep":0,"MockPercents":null,"CubeDestroyCheckFilter":false,"Debug":{"Address":""},"HttpPort":8090,"HttpBind":"127.0.0.1","Write> 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: 2026/09/20 16:24:43 migrate: 1 applied migration version(s) have no fingerprint baseline and are NOT content-checked (e.g. applied before fingerprinting existed): 0 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"LocalIp":"192.168.0.6","Module":"","@timestamp":"2026-09-20T16:24:43.38896107+08:00","Action":"","InstanceId":"","CostTime":0,"Caller":"","CallerIp":"","LogLevel":"INFO","Callee":"","RetCode":0,"InstanceType":"","FunctionType":"","LogContent":"dao schema migration completed (driver= db=cube_mvp)","Version":"","Call> 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"AppId":0,"Action":"","LocalIp":"192.168.0.6","InstanceType":"","CostTime":0,"Caller":"","InstanceId":"","LogLevel":"ERROR","RequestId":"","ContainerId":"","CodeLine":"","@timestamp":"2026-09-20T16:24:43.392978879+08:00","RetCode":0,"Version":"","Callee":"","CallerIp":"","LogContent":"templatecenter node identity: C> 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"Callee":"","LogContent":"templatecenter listen address: CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6 ignored for the listen address: http_bind=\"127.0.0.1\" is set explicitly","InstanceId":"","CallerIp":"","AppId":0,"InstanceType":"","Version":"","Caller":"","CodeLine":"","CalleeAction":"","FunctionType":"","RequestId"> 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"Namespace":"","CostTime":0,"InstanceType":"","InstanceId":"","ContainerId":"","RetCode":0,"CallerIp":"","CalleeEndpoint":"","Version":"","Action":"","LogContent":"templatecenter successfully booted in 0.014476s","LocalIp":"192.168.0.6","Module":"","CodeLine":"","RequestId":"","Caller":"","CalleeAction":"","Function> 9月 20 16:24:44 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter. [root@VM-0-6-tencentos ~]# journalctl \ -u cube-sandbox-cube-templatecenter \ -n 100 \ --no-pager 9月 20 15:36:49 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter... 9月 20 15:36:49 VM-0-6-tencentos bash[12505]: 2026-09-20T15:36:49.937762237+08:00,listen file[/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml] 9月 20 15:36:49 VM-0-6-tencentos bash[12505]: cfg:{"Common":{"MockUpdateAction":false,"DebugDumpHttpBody":false,"MockDebug":false,"MockNodeNum":0,"MockCreateDirect":false,"MockCreateDirectHandle":false,"MockHttpDirect":false,"MockCreateSleep":0,"MockPercents":null,"CubeDestroyCheckFilter":false,"Debug":{"Address":""},"HttpPort":8090,"HttpBind":"127.0.0.1","WriteTimeout":360,"ReadTimeout":120,"IdleTimeout":360,"GraceFullStopTimeoutInSec":120,"CubeOpsAddr":"","MasterAddr":"http://127.0.0.1:8089","CubeOpsBootRetries":5,"CubeOpsBootBackoff":1000000000,"SyncMetaDataInterval":1000000000,"SyncMetricDataInterval":1000000000,"CleanSandboxCacheInterval":7200000000000,"EnabledListRunningSandboxCache":false,"AsyncTaskQueueSize":10000,"AsyncTaskWorkerNum":32,"HeadlessServiceName":"","DefaultHeadlessServiceNodesNum":1,"ListFilterOutLables":null,"CollectMetricInterval":100000000,"ReportLocalCreateNum":false,"ReportStdevMetric":false,"GwCacheExpiredTime":15000000000,"GwCacheEnable":false,"ReportGWRedisGetMetric":false,"EnableGetStatusFromCubelet":false,"DisableHardDelete":false,"TemplateCacheTTL":21600000000000,"TemplateCenterAddrValue":"","CollectSandboxMemoryWhitelist":null,"EnableAllCollectSandboxMemory":false,"FilterErrMsgErrorCode":null,"DescribeInstancesWhiteList":null,"DescribeTaskExpireTime":86400,"EnablePrivateIpQuery":false,"DbMaxRetryCount":5,"DbRetryInterval":5000000,"EnableCheckComNetIDParam":false,"EnableDescribeInstanceFromRedis":false,"MaxNICQueue":4,"DisableCreateImageCluster":null,"EnableAGSColdStartSwitch":false},"AuthConf":{"Enable":false,"SignatureExpireTimeInsec":120,"SecretKeyMap":null},"Log":{"Region":"","Cluster":"","Module":"templatecenter","Path":"/data/log/CubeTemplateCenter","FileSize":100,"FileNum":10,"Level":"info","EnableLogMetric":false},"CubeletConf":{"Grpc":{"GrpcPort":9999,"CleanConnTaskIntervalInMin":60,"CleanConnTaskRoutinePoolSize":64,"ConnExpireTimeInSec":180},"CommonTimeoutInsec":30,"CreateImageTimeoutInSec":300,"AppSnapshotTimeoutInSec":300,"DefaultTimeoutInsec":0,"CreateTimeoutInsec":600,"AsyncFlows":null,"RetryCode":null,"LoopRetryCode":null,"ReuseRetryCode":null,"CircuitBreakCode":null,"ExcludeLoopRetryCode":null,"BackoffRetryCode":null,"MaxRetries":5,"LoopMaxRetries":100,"BufferQueueMinJob":10,"CreateConcurrentLimit":100,"DestroyConcurentLimit":50,"ExposedPortList":null,"EnableExposedPort":false,"DisableRedisProxyPort":false,"MaxDelayInSecond":1,"BackoffRetryDelay":5000000},"InstanceDBConfig":{"Driver":"","Addr":"127.0.0.1:3306","User":"cube","Pwd":"cube_pass","DBName":"cube_mvp","ConnTimeout":5,"ReadTimeout":5,"WriteTimeout":5,"MaxIdleConns":5,"MaxOpenConns":20,"MaxConnLifeTimeSeconds":300,"MigrationLockTimeoutSeconds":0},"RedisConf":{"Password":"ceuhvu123","MaxActive":32,"MaxIdle":8,"IdleTimeout":30,"DbNo":0,"Nodes":"127.0.0.1:6379","MaxRetry":2,"MasterName":"","SentinelNodes":"","SentinelPassword":"","NodeMetricTTLSec":600,"SandboxProxyTTLSec":0},"ExtraConf":{"BlkQos":"{}","BlkQosMap":null,"FsQos":"{}","FsQosMap":null,"NetQosList":"[]","AllowedHostMountPrefixes":null},"Scheduler":{"Overhead":{"VmMemoryOverheadBase":"42Mi","VmMemoryOverheadCoefficient":64,"HostMemoryOverheadBase":"24Mi","CubeMsgMemoryOverhead":"16Mi","VmCpuOverhead":"0","HostCpuOverhead":"0.3"},"NodeMaxMvmNum":3000,"NodeMaxMvmNumReserveNumPercent":1,"NodeMaxMemReservedInMB":10240,"NodeMaxCpuUtil":80,"PreSelectNum":-1,"PrioritySelectNum":-1,"LeastSelectName":"random","MetricUpdateTimeout":3600000000000,"LocalMetricUpdateTimeout":3600000000000,"Filter":null,"Score":null,"PostScore":null,"DisableCircuitFilter":false,"InBackoffMode":false,"AffinityConf":null,"NodeMaxMvmNumConf":null,"EnableRunInstanceHostIps":false,"MaxMvmCPU":"","MaxMvmMemory":"","DiskUsageMaxPercent":80,"LargeSizeAffinityConf":null,"NodeMaxMemReservedConf":null,"DisableBackoffFilterInstanceType":null,"ThirtpartyFilterInstanceType":null,"InstanceTypeConf":null,"NodeAffinitySelectorAllowedKeys":null,"IgnoreRedisAllocation":false,"DeprecatedOvercommitRatio":null,"DeprecatedOvercommitRatioByType":null},"ReqTemplateConf":null,"HookWhitelist":null,"CubeEgressConf":null,"CubeProxyConf":{"AdminPort":8082,"AdminToken":"","AdminURLs":null,"HeartbeatTTLMs":15000},"SoftDeletePurge":null,"VolumePlugins":null} 9月 20 15:36:49 VM-0-6-tencentos bash[12505]: 2026/09/20 15:36:49 migrate: 1 applied migration version(s) have no fingerprint baseline and are NOT content-checked (e.g. applied before fingerprinting existed): 0 9月 20 15:36:49 VM-0-6-tencentos bash[12505]: {"CallerIp":"","FunctionType":"","InstanceType":"","Action":"","CostTime":0,"AppId":0,"LogContent":"dao schema migration completed (driver= db=cube_mvp)","LocalIp":"192.168.0.6","CodeLine":"","@timestamp":"2026-09-20T15:36:49.95661439+08:00","RequestId":"","Callee":"","RetCode":0,"CalleeAction":"","CalleeEndpoint":"","Module":"","Version":"","Namespace":"","LogLevel":"INFO","ContainerId":"","Caller":"","InstanceId":""} 9月 20 15:36:49 VM-0-6-tencentos bash[12505]: {"LocalIp":"192.168.0.6","CallerIp":"","AppId":0,"RetCode":0,"InstanceType":"","RequestId":"","InstanceId":"","CostTime":0,"CalleeAction":"","Action":"","CalleeEndpoint":"","ContainerId":"","LogContent":"templatecenter node identity: CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6","Module":"","@timestamp":"2026-09-20T15:36:49.963805389+08:00","LogLevel":"ERROR","Version":"","Namespace":"","CodeLine":"","FunctionType":"","Callee":"","Caller":""} 9月 20 15:36:49 VM-0-6-tencentos bash[12505]:
从你贴出的日志看，CubeTemplateCenter 服务本身启动正常，没有启动失败问题。当前重点应该是排查 模板定义（t_cube_template_definition）为空 / 镜像模板未注册 / CubeTemplateCenter无法加载模板 这一类问题
我先分析现象
1. CubeTemplateCenter 状态正常
当前：
Active: active (running)
Main PID: 5907
启动流程：
dao schema migration completed (driver= db=cube_mvp)
templatecenter successfully booted in 0.014476s
说明：
✅ 二进制正常
✅ 配置文件可读取
✅ MySQL schema migration正常
✅ Redis配置没有阻塞
✅ 服务监听成功
2. 注意这个 WARN：监听地址问题
日志：
templatecenter listen address:
CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6 ignored
for the listen address:
http_bind="127.0.0.1" is set explicitly
你的配置：
HttpPort:8090
HttpBind:"127.0.0.1"
意味着：
127.0.0.1:8090
只能本机访问
如果 CubeMaster、CubeOps、其他节点访问 TemplateCenter，需要修改：
文件：
/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml
找到：
http_bind: 127.0.0.1
改：
http_bind: 192.168.0.6
或者：
http_bind: 0.0.0.0
然后：
systemctl restart cube-sandbox-cube-templatecenter
验证：
ss -lntp | grep 8090
应该看到：
正常：
LISTEN 0 4096 192.168.0.6:8090
或者：
LISTEN 0 4096 0.0.0.0:8090
3. t_cube_template_definition 搜索方式不对
你执行：
grep -R "t_cube_template_definition" /usr/local/services/cubetoolbox
结果：
templatecenter binary 匹配
cubemaster binary 匹配
cubeops binary 匹配
说明：
这是 Go 二进制内嵌字符串
不是数据库表
应该查 MySQL
4. 查询 Cube 数据库
日志：
DBName:"cube_mvp"
Addr:"127.0.0.1:3306"
User:"cube"
Pwd:"cube_pass"
登录：
mysql -h127.0.0.1 \
-u cube \
-pcube_pass \
cube_mvp
然后：
查看模板相关表
执行：
show tables;
重点找：
t_cube_template_definition
或者：
show tables like '%template%';
5. 查看模板定义
如果存在：
select * from t_cube_template_definition;
重点字段：
例如：
id
name
version
image
status
created_at
正常应该有类似：
cpu
gpu
ubuntu
centos
模板
6. 你现在 cube-image 只有一个镜像
你这里：
/usr/local/services/cubetoolbox/cube-image/
cube-guest-image-cpu.img 232M
version
说明：
当前只有：
CPU Guest Image
没有 GPU image
如果你要跑：
- GPU Sandbox
- NVIDIA CUDA
- AI环境
需要额外镜像，例如：
cube-guest-image-gpu.img
否则模板创建会失败
7. 检查 CubeTemplateCenter API
本机测试：
curl http://127.0.0.1:8090/
或者：
curl http://127.0.0.1:8090/health
查看：
curl -v http://127.0.0.1:8090/
8. 检查 CubeMaster
你的配置：
MasterAddr:"http://127.0.0.1:8089"
说明：
TemplateCenter依赖：
CubeMaster
|
|
TemplateCenter
|
|
Cubelet
检查：
systemctl status cube-sandbox-cube-master
以及：
ss -lntp | grep 8089
应该：
127.0.0.1:8089
9. 检查 CubeOps
配置：
CubeOpsAddr:""
但是日志没有报错
检查：
systemctl status cube-sandbox-cube-ops
10. 当前最可能问题
根据你现在的信息：
服务层
| 组件 | 状态 |
|---|---|
| CubeTemplateCenter | ✅正常 |
| MySQL | ✅正常 |
| Redis | 未知 |
| CubeMaster | 需要确认 |
| 模板数据 | 未知 |
| 镜像 | 只有CPU |
我建议下一步执行：
① 查看数据库模板
mysql -h127.0.0.1 -ucube -pcube_pass cube_mvp
执行：
show tables like '%template%';
select count(*) from t_cube_template_definition;
② 查看 TemplateCenter 注册接口
执行：
curl -s http://127.0.0.1:8090/metrics | head
③ 查看所有 Cube 服务
systemctl list-units \
| grep cube-sandbox
你目前更像是 Cube Sandbox 初始化完成，但模板中心没有加载模板镜像定义，不是 TemplateCenter 启动故障。下一步重点看 t_cube_template_definition 数据和 CubeMaster 状态
http_bind改为192.168.0.6后无法启动
[root@VM-0-6-tencentos ~]#systemctl restart cube-sandbox-cube-templatecenterJob for cube-sandbox-cube-templatecenter.service failed because the control process exited with error code.
See "systemctl status cube-sandbox-cube-templatecenter.service" and "journalctl -xeu cube-sandbox-cube-templatecenter.service" for details.
从前面的日志看，真正的问题不是 CUBE_TEMPLATE_CENTER_NODE_IP，而是 http_bind 修改后的监听行为
原配置：
HttpPort:8090
HttpBind:"127.0.0.1"
启动正常：
templatecenter successfully booted in 0.014476s
但是日志已经提示：
templatecenter listen address:
CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6 ignored for the listen address:
http_bind="127.0.0.1" is set explicitly
说明：
CUBE_TEMPLATE_CENTER_NODE_IP只影响节点身份注册- 真正 HTTP 服务监听地址由
http_bind控制
你改成：
http_bind="192.168.0.6"
后启动失败，需要进一步确认原因
1. 先查看具体失败原因
执行：
systemctl status cube-sandbox-cube-templatecenter -l
以及：
journalctl -xeu cube-sandbox-cube-templatecenter --no-pager
重点看有没有：
情况1：监听失败
例如：
bind: cannot assign requested address
说明：
192.168.0.6 并不是本机地址
确认：
ip addr | grep 192.168.0.6
应该看到：
inet 192.168.0.6/xx
情况2：端口占用
例如：
listen tcp 192.168.0.6:8090: bind: address already in use
检查：
ss -lntp | grep 8090
2. 检查 conf.yaml 修改是否正确
找到：
grep -n "http_bind" \
/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml
应该类似：
http_bind: "192.168.0.6"
注意：
不要写：
http_bind=192.168.0.6
不要写：
http_bind:192.168.0.6
3. 推荐生产配置方式
CubeSandbox 多节点环境不建议直接绑定固定 IP
推荐：
http_bind: "0.0.0.0"
即：
Common:
HttpPort: 8090
HttpBind: "0.0.0.0"
原因：
| 配置 | 效果 |
|---|---|
| 127.0.0.1 | 只能本机访问 |
| 192.168.0.6 | 只监听这个网卡 |
| 0.0.0.0 | 监听所有网卡 |
生产环境通常：
TemplateCenter
|
|
192.168.0.6:8090
|
其它Cube节点访问
应该使用：
0.0.0.0:8090
而不是：
192.168.0.6:8090
4. 修改方式
编辑：
vi /usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml
修改：
http_bind: "0.0.0.0"
然后：
systemctl daemon-reload
systemctl restart cube-sandbox-cube-templatecenter
检查：
ss -lntp | grep 8090
正常应该：
LISTEN 0 4096 0.0.0.0:8090
5. 同时检查 CubeMaster 配置
你的 TemplateCenter 配置：
MasterAddr:"http://127.0.0.1:8089"
说明当前是单机部署
如果 CubeSandbox 是多节点：
需要检查：
grep -R "TemplateCenter" \
/usr/local/services/cubetoolbox/CubeMaster/conf.yaml
以及：
grep -R "CubeTemplateCenter" \
/usr/local/services/cubetoolbox
应该类似：
http://192.168.0.6:8090
否则其它 Cubelet 无法访问模板中心
6. 你的当前环境判断
根据日志：
LocalIp":"192.168.0.6"
以及：
CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6
节点IP没有问题
当前最佳配置：
CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6
http_bind=0.0.0.0
HttpPort=8090
然后其它组件访问：
http://192.168.0.6:8090
[root@VM-0-6-tencentos ~]# systemctl status cube-sandbox-cube-templatecenter -l ● cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter Loaded: loaded (/etc/systemd/system/cube-sandbox-cube-templatecenter.service; enabled; preset: disabled) Active: activating (auto-restart) (Result: exit-code) since Sun 2026-09-20 18:38:43 CST; 1s ago Process: 240327 ExecStart=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubetemplatecenter-start.sh (code=exited, status=0/SUCCESS) Process: 240328 ExecStartPost=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubetemplatecenter-postcheck.sh (code=exited, status=1/FAILURE) Main PID: 240327 (code=exited, status=0/SUCCESS) CPU: 281ms [root@VM-0-6-tencentos ~]# journalctl -xeu cube-sandbox-cube-templatecenter --no-pager 9月 20 16:24:43 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter... ░░ Subject: cube-sandbox-cube-templatecenter.service 单元已开始启动 ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ cube-sandbox-cube-templatecenter.service 单元已开始启动。 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: 2026-09-20T16:24:43.375476072+08:00,listen file[/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml] 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: cfg:{"Common":{"MockUpdateAction":false,"DebugDumpHttpBody":false,"MockDebug":false,"MockNodeNum":0,"MockCreateDirect":false,"MockCreateDirectHandle":false,"MockHttpDirect":false,"MockCreateSleep":0,"MockPercents":null,"CubeDestroyCheckFilter":false,"Debug":{"Address":""},"HttpPort":8090,"HttpBind":"127.0.0.1","WriteTimeout":360,"ReadTimeout":120,"IdleTimeout":360,"GraceFullStopTimeoutInSec":120,"CubeOpsAddr":"","MasterAddr":"http://127.0.0.1:8089","CubeOpsBootRetries":5,"CubeOpsBootBackoff":1000000000,"SyncMetaDataInterval":1000000000,"SyncMetricDataInterval":1000000000,"CleanSandboxCacheInterval":7200000000000,"EnabledListRunningSandboxCache":false,"AsyncTaskQueueSize":10000,"AsyncTaskWorkerNum":32,"HeadlessServiceName":"","DefaultHeadlessServiceNodesNum":1,"ListFilterOutLables":null,"CollectMetricInterval":100000000,"ReportLocalCreateNum":false,"ReportStdevMetric":false,"GwCacheExpiredTime":15000000000,"GwCacheEnable":false,"ReportGWRedisGetMetric":false,"EnableGetStatusFromCubelet":false,"DisableHardDelete":false,"TemplateCacheTTL":21600000000000,"TemplateCenterAddrValue":"","CollectSandboxMemoryWhitelist":null,"EnableAllCollectSandboxMemory":false,"FilterErrMsgErrorCode":null,"DescribeInstancesWhiteList":null,"DescribeTaskExpireTime":86400,"EnablePrivateIpQuery":false,"DbMaxRetryCount":5,"DbRetryInterval":5000000,"EnableCheckComNetIDParam":false,"EnableDescribeInstanceFromRedis":false,"MaxNICQueue":4,"DisableCreateImageCluster":null,"EnableAGSColdStartSwitch":false},"AuthConf":{"Enable":false,"SignatureExpireTimeInsec":120,"SecretKeyMap":null},"Log":{"Region":"","Cluster":"","Module":"templatecenter","Path":"/data/log/CubeTemplateCenter","FileSize":100,"FileNum":10,"Level":"info","EnableLogMetric":false},"CubeletConf":{"Grpc":{"GrpcPort":9999,"CleanConnTaskIntervalInMin":60,"CleanConnTaskRoutinePoolSize":64,"ConnExpireTimeInSec":180},"CommonTimeoutInsec":30,"CreateImageTimeoutInSec":300,"AppSnapshotTimeoutInSec":300,"DefaultTimeoutInsec":0,"CreateTimeoutInsec":600,"AsyncFlows":null,"RetryCode":null,"LoopRetryCode":null,"ReuseRetryCode":null,"CircuitBreakCode":null,"ExcludeLoopRetryCode":null,"BackoffRetryCode":null,"MaxRetries":5,"LoopMaxRetries":100,"BufferQueueMinJob":10,"CreateConcurrentLimit":100,"DestroyConcurentLimit":50,"ExposedPortList":null,"EnableExposedPort":false,"DisableRedisProxyPort":false,"MaxDelayInSecond":1,"BackoffRetryDelay":5000000},"InstanceDBConfig":{"Driver":"","Addr":"127.0.0.1:3306","User":"cube","Pwd":"cube_pass","DBName":"cube_mvp","ConnTimeout":5,"ReadTimeout":5,"WriteTimeout":5,"MaxIdleConns":5,"MaxOpenConns":20,"MaxConnLifeTimeSeconds":300,"MigrationLockTimeoutSeconds":0},"RedisConf":{"Password":"ceuhvu123","MaxActive":32,"MaxIdle":8,"IdleTimeout":30,"DbNo":0,"Nodes":"127.0.0.1:6379","MaxRetry":2,"MasterName":"","SentinelNodes":"","SentinelPassword":"","NodeMetricTTLSec":600,"SandboxProxyTTLSec":0},"ExtraConf":{"BlkQos":"{}","BlkQosMap":null,"FsQos":"{}","FsQosMap":null,"NetQosList":"[]","AllowedHostMountPrefixes":null},"Scheduler":{"Overhead":{"VmMemoryOverheadBase":"42Mi","VmMemoryOverheadCoefficient":64,"HostMemoryOverheadBase":"24Mi","CubeMsgMemoryOverhead":"16Mi","VmCpuOverhead":"0","HostCpuOverhead":"0.3"},"NodeMaxMvmNum":3000,"NodeMaxMvmNumReserveNumPercent":1,"NodeMaxMemReservedInMB":10240,"NodeMaxCpuUtil":80,"PreSelectNum":-1,"PrioritySelectNum":-1,"LeastSelectName":"random","MetricUpdateTimeout":3600000000000,"LocalMetricUpdateTimeout":3600000000000,"Filter":null,"Score":null,"PostScore":null,"DisableCircuitFilter":false,"InBackoffMode":false,"AffinityConf":null,"NodeMaxMvmNumConf":null,"EnableRunInstanceHostIps":false,"MaxMvmCPU":"","MaxMvmMemory":"","DiskUsageMaxPercent":80,"LargeSizeAffinityConf":null,"NodeMaxMemReservedConf":null,"DisableBackoffFilterInstanceType":null,"ThirtpartyFilterInstanceType":null,"InstanceTypeConf":null,"NodeAffinitySelectorAllowedKeys":null,"IgnoreRedisAllocation":false,"DeprecatedOvercommitRatio":null,"DeprecatedOvercommitRatioByType":null},"ReqTemplateConf":null,"HookWhitelist":null,"CubeEgressConf":null,"CubeProxyConf":{"AdminPort":8082,"AdminToken":"","AdminURLs":null,"HeartbeatTTLMs":15000},"SoftDeletePurge":null,"VolumePlugins":null} 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: 2026/09/20 16:24:43 migrate: 1 applied migration version(s) have no fingerprint baseline and are NOT content-checked (e.g. applied before fingerprinting existed): 0 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"LocalIp":"192.168.0.6","Module":"","@timestamp":"2026-09-20T16:24:43.38896107+08:00","Action":"","InstanceId":"","CostTime":0,"Caller":"","CallerIp":"","LogLevel":"INFO","Callee":"","RetCode":0,"InstanceType":"","FunctionType":"","LogContent":"dao schema migration completed (driver= db=cube_mvp)","Version":"","CalleeEndpoint":"","RequestId":"","AppId":0,"CodeLine":"","Namespace":"","CalleeAction":"","ContainerId":""} 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"AppId":0,"Action":"","LocalIp":"192.168.0.6","InstanceType":"","CostTime":0,"Caller":"","InstanceId":"","LogLevel":"ERROR","RequestId":"","ContainerId":"","CodeLine":"","@timestamp":"2026-09-20T16:24:43.392978879+08:00","RetCode":0,"Version":"","Callee":"","CallerIp":"","LogContent":"templatecenter node identity: CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6","Module":"","CalleeAction":"","Namespace":"","FunctionType":"","CalleeEndpoint":""} 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"Callee":"","LogContent":"templatecenter listen address: CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6 ignored for the listen address: http_bind=\"127.0.0.1\" is set explicitly","InstanceId":"","CallerIp":"","AppId":0,"InstanceType":"","Version":"","Caller":"","CodeLine":"","CalleeAction":"","FunctionType":"","RequestId":"","CostTime":0,"LocalIp":"192.168.0.6","Module":"","LogLevel":"WARN","ContainerId":"","RetCode":0,"Namespace":"","CalleeEndpoint":"","@timestamp":"2026-09-20T16:24:43.393017301+08:00","Action":""} 9月 20 16:24:43 VM-0-6-tencentos bash[5907]: {"Namespace":"","CostTime":0,"InstanceType":"","InstanceId":"","ContainerId":"","RetCode":0,"CallerIp":"","CalleeEndpoint":"","Version":"","Action":"","LogContent":"templatecenter successfully booted in 0.014476s","LocalIp":"192.168.0.6","Module":"","CodeLine":"","RequestId":"","Caller":"","CalleeAction":"","FunctionType":"","@timestamp":"2026-09-20T16:24:43.394146817+08:00","LogLevel":"ERROR","AppId":0,"Callee":""} 9月 20 16:24:44 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter. ░░ Subject: cube-sandbox-cube-templatecenter.service 单元已结束启动 ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ cube-sandbox-cube-templatecenter.service 单元已结束启动。 ░░ ░░ 启动结果为“done”。 9月 20 18:28:05 VM-0-6-tencentos bash[5907]: 2026-09-20T18:28:05.383136309+08:00,OnFileEvent:WRITE "/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml" 9月 20 18:28:05 VM-0-6-tencentos bash[5907]: setLogLevel:info 9月 20 18:28:05 VM-0-6-tencentos bash[5907]: 2026-09-20T18:28:05.384444503+08:00,OnFileEvent:CHMOD "/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml" 9月 20 18:28:06 VM-0-6-tencentos bash[5907]: {"CostTime":0,"LogLevel":"INFO","RetCode":0,"AppId":0,"Module":"","FunctionType":"","InstanceId":"","Caller":"","CalleeAction":"","ContainerId":"","Version":"","LogContent":"templatecenter received signal terminated, shutting down","LocalIp":"192.168.0.6","@timestamp":"2026-09-20T18:28:06.568537334+08:00","Namespace":"","RequestId":"","InstanceType":"","CallerIp":"","Callee":"","CodeLine":"","CalleeEndpoint":"","Action":""} 9月 20 18:28:06 VM-0-6-tencentos bash[5907]: {"Caller":"","LogLevel":"ERROR","Callee":"","ContainerId":"","AppId":0,"InstanceId":"","CalleeEndpoint":"","CalleeAction":"","LogContent":"templatecenter shutting down: cancelling in-flight builds","FunctionType":"","CostTime":0,"LocalIp":"192.168.0.6","Module":"","@timestamp":"2026-09-20T18:28:06.568596615+08:00","Namespace":"","RequestId":"","Version":"","RetCode":0,"InstanceType":"","CodeLine":"","Action":"","CallerIp":""} 9月 20 18:28:06 VM-0-6-tencentos bash[5907]: {"@timestamp":"2026-09-20T18:28:06.568607436+08:00","LogLevel":"ERROR","LogContent":"templatecenter stopped gracefully begin, pid 5907","LocalIp":"192.168.0.6","Module":"","CodeLine":""} 9月 20 18:28:06 VM-0-6-tencentos bash[5907]: {"CodeLine":"","@timestamp":"2026-09-20T18:28:06.56866807+08:00","LogLevel":"ERROR","LogContent":"templatecenter InternalHttp Shutdown succ","LocalIp":"192.168.0.6","Module":""} 9月 20 18:28:06 VM-0-6-tencentos systemd[1]: Stopping cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter... ░░ Subject: cube-sandbox-cube-templatecenter.service 单元已开始停止操作 ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ cube-sandbox-cube-templatecenter.service 单元已开始停止操作。 9月 20 18:28:06 VM-0-6-tencentos systemd[1]: cube-sandbox-cube-templatecenter.service: Deactivated successfully. ░░ Subject: Unit succeeded ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ The unit cube-sandbox-cube-templatecenter.service has successfully entered the 'dead' state. 9月 20 18:28:06 VM-0-6-tencentos systemd[1]: Stopped cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter. ░░ Subject: cube-sandbox-cube-templatecenter.service 单元已结束停止操作 ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ cube-sandbox-cube-templatecenter.service 单元已结束停止操作。 9月 20 18:28:06 VM-0-6-tencentos systemd[1]: cube-sandbox-cube-templatecenter.service: Consumed 10.129s CPU time, 48.5M memory peak, 0B memory swap peak. ░░ Subject: Resources consumed by unit runtime ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ The unit cube-sandbox-cube-templatecenter.service completed and consumed the indicated resources. 9月 20 18:28:06 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter... ░░ Subject: cube-sandbox-cube-templatecenter.service 单元已开始启动 ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ cube-sandbox-cube-templatecenter.service 单元已开始启动。 9月 20 18:28:06 VM-0-6-tencentos bash[220740]: 2026-09-20T18:28:06.627410234+08:00,listen file[/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml] 9月 20 18:28:06 VM-0-6-tencentos bash[220740]: cfg:{"Common":{"MockUpdateAction":false,"DebugDumpHttpBody":false,"MockDebug":false,"MockNodeNum":0,"MockCreateDirect":false,"MockCreateDirectHandle":false,"MockHttpDirect":false,"MockCreateSleep":0,"MockPercents":null,"CubeDestroyCheckFilter":false,"Debug":{"Address":""},"HttpPort":8090,"HttpBind":"0.0.0.0","WriteTimeout":360,"ReadTimeout":120,"IdleTimeout":360,"GraceFullStopTimeoutInSec":120,"CubeOpsAddr":"","MasterAddr":"http://192.168.0.6:8089","CubeOpsBootRetries":5,"CubeOpsBootBackoff":1000000000,"SyncMetaDataInterval":1000000000,"SyncMetricDataInterval":1000000000,"CleanSandboxCacheInterval":7200000000000,"EnabledListRunningSandboxCache":false,"AsyncTaskQueueSize":10000,"AsyncTaskWorkerNum":32,"HeadlessServiceName":"","DefaultHeadlessServiceNodesNum":1,"ListFilterOutLables":null,"CollectMetricInterval":100000000,"ReportLocalCreateNum":false,"ReportStdevMetric":false,"GwCacheExpiredTime":15000000000,"GwCacheEnable":false,"ReportGWRedisGetMetric":false,"EnableGetStatusFromCubelet":false,"DisableHardDelete":false,"TemplateCacheTTL":21600000000000,"TemplateCenterAddrValue":"","CollectSandboxMemoryWhitelist":null,"EnableAllCollectSandboxMemory":false,"FilterErrMsgErrorCode":null,"DescribeInstancesWhiteList":null,"DescribeTaskExpireTime":86400,"EnablePrivateIpQuery":false,"DbMaxRetryCount":5,"DbRetryInterval":5000000,"EnableCheckComNetIDParam":false,"EnableDescribeInstanceFromRedis":false,"MaxNICQueue":4,"DisableCreateImageCluster":null,"EnableAGSColdStartSwitch":false},"AuthConf":{"Enable":false,"SignatureExpireTimeInsec":120,"SecretKeyMap":null},"Log":{"Region":"","Cluster":"","Module":"templatecenter","Path":"/data/log/CubeTemplateCenter","FileSize":100,"FileNum":10,"Level":"info","EnableLogMetric":false},"CubeletConf":{"Grpc":{"GrpcPort":9999,"CleanConnTaskIntervalInMin":60,"CleanConnTaskRoutinePoolSize":64,"ConnExpireTimeInSec":180},"CommonTimeoutInsec":30,"CreateImageTimeoutInSec":300,"AppSnapshotTimeoutInSec":300,"DefaultTimeoutInsec":0,"CreateTimeoutInsec":600,"AsyncFlows":null,"RetryCode":null,"LoopRetryCode":null,"ReuseRetryCode":null,"CircuitBreakCode":null,"ExcludeLoopRetryCode":null,"BackoffRetryCode":null,"MaxRetries":5,"LoopMaxRetries":100,"BufferQueueMinJob":10,"CreateConcurrentLimit":100,"DestroyConcurentLimit":50,"ExposedPortList":null,"EnableExposedPort":false,"DisableRedisProxyPort":false,"MaxDelayInSecond":1,"BackoffRetryDelay":5000000},"InstanceDBConfig":{"Driver":"","Addr":"127.0.0.1:3306","User":"cube","Pwd":"cube_pass","DBName":"cube_mvp","ConnTimeout":5,"ReadTimeout":5,"WriteTimeout":5,"MaxIdleConns":5,"MaxOpenConns":20,"MaxConnLifeTimeSeconds":300,"MigrationLockTimeoutSeconds":0},"RedisConf":{"Password":"ceuhvu123","MaxActive":32,"MaxIdle":8,"IdleTimeout":30,"DbNo":0,"Nodes":"127.0.0.1:6379","MaxRetry":2,"MasterName":"","SentinelNodes":"","SentinelPassword":"","NodeMetricTTLSec":600,"SandboxProxyTTLSec":0},"ExtraConf":{"BlkQos":"{}","BlkQosMap":null,"FsQos":"{}","FsQosMap":null,"NetQosList":"[]","AllowedHostMountPrefixes":null},"Scheduler":{"Overhead":{"VmMemoryOverheadBase":"42Mi","VmMemoryOverheadCoefficient":64,"HostMemoryOverheadBase":"24Mi","CubeMsgMemoryOverhead":"16Mi","VmCpuOverhead":"0","HostCpuOverhead":"0.3"},"NodeMaxMvmNum":3000,"NodeMaxMvmNumReserveNumPercent":1,"NodeMaxMemReservedInMB":10240,"NodeMaxCpuUtil":80,"PreSelectNum":-1,"PrioritySelectNum":-1,"LeastSelectName":"random","MetricUpdateTimeout":3600000000000,"LocalMetricUpdateTimeout":3600000000000,"Filter":null,"Score":null,"PostScore":null,"DisableCircuitFilter":false,"InBackoffMode":false,"AffinityConf":null,"NodeMaxMvmNumConf":null,"EnableRunInstanceHostIps":false,"MaxMvmCPU":"","MaxMvmMemory":"","DiskUsageMaxPercent":80,"LargeSizeAffinityConf":null,"NodeMaxMemReservedConf":null,"DisableBackoffFilterInstanceType":null,"ThirtpartyFilterInstanceType":null,"InstanceTypeConf":null,"NodeAffinitySelectorAllowedKeys":null,"IgnoreRedisAllocation":false,"DeprecatedOvercommitRatio":null,"DeprecatedOvercommitRatioByType":null},"ReqTemplateConf":null,"HookWhitelist":null,"CubeEgressConf":null,"CubeProxyConf":{"AdminPort":8082,"AdminToken":"","AdminURLs":null,"HeartbeatTTLMs":15000},"SoftDeletePurge":null,"VolumePlugins":null} 9月 20 18:28:06 VM-0-6-tencentos bash[220740]: 2026/09/20 18:28:06 migrate: 1 applied migration version(s) have no fingerprint baseline and are NOT content-checked (e.g. applied before fingerprinting existed): 0 9月 20 18:28:06 VM-0-6-tencentos bash[220740]: {"Action":"","Version":"","CallerIp":"","CalleeAction":"","LogContent":"dao schema migration completed (driver= db=cube_mvp)","CodeLine":"","@timestamp":"2026-09-20T18:28:06.63496995+08:00","LogLevel":"INFO","Callee":"","FunctionType":"","AppId":0,"ContainerId":"","InstanceId":"","Module":"","InstanceType":"","RetCode":0,"RequestId":"","LocalIp":"192.168.0.6","CalleeEndpoint":"","CostTime":0,"Namespace":"","Caller":""} 9月 20 18:28:06 VM-0-6-tencentos bash[220740]: {"Module":"","AppId":0,"Namespace":"","ContainerId":"","Caller":"","CodeLine":"","InstanceId":"","RetCode":0,"CostTime":0,"RequestId":"","Action":"","CalleeEndpoint":"","FunctionType":"","@timestamp":"2026-09-20T18:28:06.637522696+08:00","LogLevel":"ERROR","Version":"","CalleeAction":"","Callee":"","LogContent":"templatecenter node identity: CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6","LocalIp":"192.168.0.6","CallerIp":"","InstanceType":""} 9月 20 18:28:06 VM-0-6-tencentos bash[220740]: {"Callee":"","RequestId":"","Version":"","InstanceType":"","InstanceId":"","Action":"","CostTime":0,"CodeLine":"","@timestamp":"2026-09-20T18:28:06.637677728+08:00","LogLevel":"WARN","AppId":0,"Caller":"","Namespace":"","RetCode":0,"LogContent":"templatecenter listen address: listening on 192.168.0.6 from CUBE_TEMPLATE_CENTER_NODE_IP","LocalIp":"192.168.0.6","Module":"","ContainerId":"","CalleeEndpoint":"","CalleeAction":"","CallerIp":"","FunctionType":""} 9月 20 18:28:06 VM-0-6-tencentos bash[220740]: {"Module":"","CodeLine":"","@timestamp":"2026-09-20T18:28:06.637922399+08:00","LogLevel":"ERROR","CostTime":0,"Caller":"","FunctionType":"","AppId":0,"LocalIp":"192.168.0.6","Namespace":"","CallerIp":"","CalleeEndpoint":"","Version":"","InstanceId":"","InstanceType":"","RequestId":"","LogContent":"templatecenter successfully booted in 0.009064s","Callee":"","Action":"","RetCode":0,"CalleeAction":"","ContainerId":""} 9月 20 18:28:36 VM-0-6-tencentos bash[220741]: [one-click-systemd] ERROR wait_for_http timeout: url=http://127.0.0.1:8090/health waited=30s last_curl_error=curl: (7) Failed to connect to 127.0.0.1 port 8090 after 0 ms: Couldn't connect to server 9月 20 18:28:36 VM-0-6-tencentos bash[220741]: [one-click-systemd] ERROR: cubetemplatecenter health not ready 9月 20 18:28:36 VM-0-6-tencentos systemd[1]: cube-sandbox-cube-templatecenter.service: Control process exited, code=exited, status=1/FAILURE ░░ Subject: Unit process exited ░░ Defined-By: systemd ░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel ░░ ░░ An ExecStartPost= process belonging to unit cube-sandbox-cube-templatecenter.service has exited. ░░ ░░ The process' exit code is 'exited' and its exit status is 1. 9月 20 18:28:36 VM-0-6-tencentos bash[220740]: {"@timestamp":"2026-09-20T18:28:36.777787861+08:00","AppId":0,"Action":"","CostTime":0,"LogLevel":"INFO","InstanceType":"","LogContent":"templatecenter received signal terminated, shutting down","Module":"","CodeLine":"","InstanceId":"","ContainerId":"","CallerIp":"","Caller":"","Version":"","Namespace":"","LocalIp":"192.168.0.6","CalleeAction":"","RequestId":"","FunctionType":"","Callee":"","RetCode":0,"CalleeEndpoint":""} 9月 20 18:28:36 VM-0-6-tencentos bash[220740]: {"LogLevel":"ERROR","RetCode":0,"AppId":0,"Caller":"","Callee":"","CostTime":0,"Module":"","FunctionType":"","InstanceType":"","Version":"","LocalIp":"192.168.0.6","CodeLine":"","Namespace":"","ContainerId":"","InstanceId":"","@timestamp":"2026-09-20T18:28:36.777843506+08:00","Action":"","RequestId":"","CalleeEndpoint":"","CallerIp":"","CalleeAction":"","LogContent":"templatecenter shutting down: cancelling in-flight builds"} 9月 20 18:28:36 VM-0-6-tencentos bash[220740
从日志看，CubeTemplateCenter 本身启动成功，失败点在 systemd 的健康检查脚本，不是程序启动失败
关键日志：
templatecenter successfully booted in 0.009064s
说明服务已经起来
但是随后：
[one-click-systemd] ERROR wait_for_http timeout:
url=http://127.0.0.1:8090/health
curl: (7) Failed to connect to 127.0.0.1 port 8090
然后：
ExecStartPost=/usr/bin/bash ... cubetemplatecenter-postcheck.sh
status=1/FAILURE
systemd 认为启动失败，于是发送停止信号：
templatecenter received signal terminated, shutting down
根因
你修改了：
原来：
HttpBind: "127.0.0.1"
改成：
HttpBind: "192.168.0.6"
或者：
HttpBind: "0.0.0.0"
程序监听地址变了
但是健康检查脚本仍然固定检查：
curl http://127.0.0.1:8090/health
所以：
程序：
192.168.0.6:8090
监听
健康检查：
127.0.0.1:8090
访问不到
验证
执行：
ss -lntp | grep 8090
你应该看到类似：
当前配置 192.168.0.6
LISTEN 0 4096 192.168.0.6:8090
或者：
LISTEN 0 4096 0.0.0.0:8090
测试：
curl http://192.168.0.6:8090/health
应该正常
但是：
curl http://127.0.0.1:8090/health
失败
解决方案 1（推荐）：修改 postcheck 健康检查地址
查看：
cat /usr/local/services/cubetoolbox/scripts/systemd/cubetemplatecenter-postcheck.sh
应该类似：
wait_for_http http://127.0.0.1:8090/health
改：
wait_for_http http://192.168.0.6:8090/health
或者更推荐：
使用环境变量：
CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6
例如：
wait_for_http http://${CUBE_TEMPLATE_CENTER_NODE_IP}:8090/health
然后：
systemctl daemon-reload
systemctl restart cube-sandbox-cube-templatecenter
解决方案 2（更推荐生产）：监听 0.0.0.0 + localhost健康检查
生产环境建议：
conf.yaml:
Common:
HttpBind: "0.0.0.0"
HttpPort: 8090
保持：
curl http://127.0.0.1:8090/health
不变
原因：
- 服务监听所有网卡
- 本机健康检查稳定
- 避免绑定具体 IP 后 IP变化导致故障
即：
+-------------+
192.168.0.6 ------> | |
| TemplateCtr |
127.0.0.1 --------> | |
+-------------+
:8090
解决方案 3：增加127.0.0.1绑定（不推荐）
如果程序支持多监听，可以：
HttpBind: "0.0.0.0"
不要写：
127.0.0.1,192.168.0.6
很多 Go HTTP 服务不支持
另外检查一个隐藏问题
你的配置里：
MasterAddr":"http://192.168.0.6:8089"
之前：
MasterAddr":"http://127.0.0.1:8089"
如果 CubeMaster 也监听：
ss -lntp | grep 8089
确认：
192.168.0.6:8089
或者：
0.0.0.0:8089
否则 TemplateCenter 后续可能连接 Master 失败
建议最终生产配置
CubeTemplateCenter/conf.yaml
Common:
HttpBind: "0.0.0.0"
HttpPort: 8090
MasterAddr: "http://192.168.0.6:8089"
环境：
export CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6
systemd:
ExecStartPost=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubetemplatecenter-postcheck.sh
postcheck:
curl -f http://127.0.0.1:8090/health
然后：
systemctl daemon-reload
systemctl restart cube-sandbox-cube-templatecenter
systemctl status cube-sandbox-cube-templatecenter -l
[root@VM-0-6-tencentos ~]# systemctl status cube-sandbox-cube-templatecenter -l ● cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter Loaded: loaded (/etc/systemd/system/cube-sandbox-cube-templatecenter.service; enabled; preset: disabled) Active: active (running) since Sun 2026-09-20 18:47:52 CST; 34min ago Process: 258184 ExecStartPost=/usr/bin/bash /usr/local/services/cubetoolbox/scripts/systemd/cubetemplatecenter-postcheck.sh (code=exited, status=0/SUCCESS) Main PID: 258182 (templatecenter) Tasks: 25 (limit: 153562) Memory: 11.8M (peak: 13.4M) CPU: 2.806s CGroup: /system.slice/cube-sandbox-cube-templatecenter.service └─258182 /usr/local/services/cubetoolbox/CubeTemplateCenter/bin/templatecenter 9月 20 18:47:51 VM-0-6-tencentos systemd[1]: Starting cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter... 9月 20 18:47:51 VM-0-6-tencentos bash[258182]: 2026-09-20T18:47:51.806270415+08:00,listen file[/usr/local/services/cubetoolbox/CubeTemplateCenter/conf.yaml] 9月 20 18:47:51 VM-0-6-tencentos bash[258182]: cfg:{"Common":{"MockUpdateAction":false,"DebugDumpHttpBody":false,"MockDebug":false,"MockNodeNum":0,"MockCreateDirect":false,"MockCreateDirectHandle":false,"MockHttpDirect":false,"MockCreateSleep":0,"MockPercents":null,"CubeDestroyCheckFilter":false,"Debug":{"Address":""},"HttpPort":8090,"HttpBind":"0.0.0.0","Write> 9月 20 18:47:51 VM-0-6-tencentos bash[258182]: 2026/09/20 18:47:51 migrate: 1 applied migration version(s) have no fingerprint baseline and are NOT content-checked (e.g. applied before fingerprinting existed): 0 9月 20 18:47:51 VM-0-6-tencentos bash[258182]: {"InstanceId":"","LogContent":"dao schema migration completed (driver= db=cube_mvp)","Module":"","Caller":"","AppId":0,"Namespace":"","CostTime":0,"CallerIp":"","CalleeEndpoint":"","Action":"","LocalIp":"192.168.0.6","CodeLine":"","@timestamp":"2026-09-20T18:47:51.813844537+08:00","RequestId":"","ContainerId":"","C> 9月 20 18:47:51 VM-0-6-tencentos bash[258182]: {"LogLevel":"ERROR","InstanceId":"","LogContent":"templatecenter node identity: CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6","LocalIp":"192.168.0.6","CodeLine":"","@timestamp":"2026-09-20T18:47:51.815869199+08:00","AppId":0,"Namespace":"","RequestId":"","Callee":"","Module":"","CalleeAction":"","Action":"","RetCode":0> 9月 20 18:47:51 VM-0-6-tencentos bash[258182]: {"LocalIp":"192.168.0.6","LogLevel":"WARN","InstanceId":"","ContainerId":"","AppId":0,"FunctionType":"","Callee":"","CalleeEndpoint":"","Namespace":"","Action":"","LogContent":"templatecenter listen address: listening on 192.168.0.6 from CUBE_TEMPLATE_CENTER_NODE_IP","Module":"","CalleeAction":"","CostTime":0,"Call> 9月 20 18:47:51 VM-0-6-tencentos bash[258182]: {"Caller":"","LogContent":"templatecenter successfully booted in 0.008396s","LogLevel":"ERROR","AppId":0,"RequestId":"","CallerIp":"","CodeLine":"","Action":"","RetCode":0,"Version":"","InstanceId":"","ContainerId":"","CalleeAction":"","Callee":"","Module":"","CalleeEndpoint":"","InstanceType":"","FunctionType":"",> 9月 20 18:47:52 VM-0-6-tencentos systemd[1]: Started cube-sandbox-cube-templatecenter.service - Cube Sandbox CubeTemplateCenter.
现在状态已经正常
关键状态：
Active: active (running) since Sun 2026-09-20 18:47:52 CST
说明：
- ✅ CubeTemplateCenter 进程正常运行
- ✅ systemd 启动成功
- ✅ ExecStartPost 健康检查通过
- ✅ 不再因为
/health检查失败自动退出
之前失败：
curl http://127.0.0.1:8090/health
Failed to connect
现在：
ExecStartPost ... cubetemplatecenter-postcheck.sh
(code=exited, status=0/SUCCESS)
说明健康检查恢复
当前监听模式确认
你现在配置：
Common:
HttpBind: "0.0.0.0"
HttpPort: 8090
日志：
templatecenter listen address:
listening on 192.168.0.6 from CUBE_TEMPLATE_CENTER_NODE_IP
说明：
- 服务绑定所有网卡
- 对外地址使用
192.168.0.6 - 节点身份使用
CUBE_TEMPLATE_CENTER_NODE_IP
这是比较适合生产部署的方式
建议再做几个确认：
1. 查看监听
ss -lntp | grep 8090
预期：
LISTEN 0 4096 0.0.0.0:8090
或者：
LISTEN 0 4096 *:8090
2. 本机健康检查
curl -v http://127.0.0.1:8090/health
应该返回类似：
{
"status":"ok"
}
3. 外部节点访问
从其他 Cube 节点：
curl http://192.168.0.6:8090/health
确认网络连通
4. 检查 CubeMaster 地址
你之前改过：
MasterAddr: "http://192.168.0.6:8089"
确认 Master：
ss -lntp | grep 8089
以及：
curl http://192.168.0.6:8089/health
避免 TemplateCenter 启动正常，但后续注册 Master 失败
推荐最终配置
CubeTemplateCenter/conf.yaml
保持：
Common:
HttpBind: "0.0.0.0"
HttpPort: 8090
MasterAddr: "http://192.168.0.6:8089"
环境：
CUBE_TEMPLATE_CENTER_NODE_IP=192.168.0.6
当前这个节点已经恢复，可以继续验证 CubeOps / Cubelet 注册状态。建议下一步检查：
grep -i "error\|failed\|register" /data/log/CubeTemplateCenter/*.log
确认 TemplateCenter 是否已经成功向 CubeMaster 注册
[root@VM-0-6-tencentos ~]# grep -i "error\|failed\|register" /data/log/CubeTemplateCenter/*.log {"Module":"templatecenter","LogLevel":"WARN","Version":"v0.7.1","CallerIp":"","InstanceType":"","CodeLine":"","Callee":"","ContainerId":"","RequestId":"","@timestamp":"2026-09-20T17:19:28.408026104+08:00","CalleeAction":"","CostTime":0,"AppId":0,"RetCode":0,"LogContent":"report status attempt 1/6 failed (retrying in 500ms): job_id=2bbfb535-ff16-489b-a827-596b81337cf5 err=master returned 500: {\"error\":\"record not found\"}","LocalIp":"192.168.0.6","Namespace":"","Action":"","Caller":"","CalleeEndpoint":"","InstanceId":"","FunctionType":""} {"CodeLine":"","@timestamp":"2026-09-20T17:19:28.912537795+08:00","LogLevel":"ERROR","template_id":"tpl-57680aeb828e4b6fb3fde0ba","AppId":0,"CostTime":0,"CalleeAction":"","Action":"","Module":"templatecenter","Version":"v0.7.1","image":"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2","RequestId":"","CalleeEndpoint":"","Callee":"","Namespace":"","Caller":"","InstanceType":"","ContainerId":"","job_id":"2bbfb535-ff16-489b-a827-596b81337cf5","RetCode":0,"LocalIp":"192.168.0.6","InstanceId":"","FunctionType":"","CallerIp":"","LogContent":"pull image fail: native export failed to resolve image \"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2\" (if this is a multi-arch index, verify the requested platform exists): Get \"https://harbor-intranet.allintechinc.com/v2/\": dial tcp: lookup harbor-intranet.allintechinc.com: no such host"} {"ContainerId":"","Caller":"","CalleeAction":"","RetCode":0,"InstanceType":"","Action":"","LocalIp":"192.168.0.6","CalleeEndpoint":"","CallerIp":"","CostTime":0,"RequestId":"","Module":"templatecenter","InstanceId":"","Version":"v0.7.1","AppId":0,"Callee":"","Namespace":"","LogLevel":"INFO","FunctionType":"","LogContent":"report status to master: job_id=2bbfb535-ff16-489b-a827-596b81337cf5 url=http://127.0.0.1:8089/internal/template/jobs/2bbfb535-ff16-489b-a827-596b81337cf5/status fields=map[error_message:pull image fail: native export failed to resolve image \"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2\" (if this is a multi-arch index, verify the requested platform exists): Get \"https://harbor-intranet.allintechinc.com/v2/\": dial tcp: lookup harbor-intranet.allintechinc.com: no such host phase:PULLING progress:100 status:FAILED]","CodeLine":"","@timestamp":"2026-09-20T17:19:28.912595103+08:00"} {"CostTime":0,"InstanceId":"","Caller":"","LocalIp":"192.168.0.6","LogLevel":"ERROR","Namespace":"","InstanceType":"","AppId":0,"CalleeEndpoint":"","CodeLine":"","CalleeAction":"","FunctionType":"","Version":"v0.7.1","Action":"","RequestId":"","LogContent":"build template fail: job_id=2bbfb535-ff16-489b-a827-596b81337cf5 err=pull image: native export failed to resolve image \"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2\" (if this is a multi-arch index, verify the requested platform exists): Get \"https://harbor-intranet.allintechinc.com/v2/\": dial tcp: lookup harbor-intranet.allintechinc.com: no such host","Module":"templatecenter","@timestamp":"2026-09-20T17:19:28.916360382+08:00","CallerIp":"","Callee":"","RetCode":0,"ContainerId":""} {"LogLevel":"WARN","Callee":"","Caller":"","Namespace":"","LocalIp":"192.168.0.6","CodeLine":"","AppId":0,"CalleeAction":"","InstanceId":"","CalleeEndpoint":"","LogContent":"report status attempt 1/6 failed (retrying in 500ms): job_id=00306852-77df-4edd-895e-d302a7dbad29 err=master returned 500: {\"error\":\"record not found\"}","Module":"templatecenter","CallerIp":"","CostTime":0,"Action":"","Version":"v0.7.1","@timestamp":"2026-09-20T17:28:33.330953795+08:00","RetCode":0,"ContainerId":"","FunctionType":"","InstanceType":"","RequestId":""} {"Caller":"","ContainerId":"","Module":"templatecenter","LogLevel":"ERROR","InstanceType":"","RequestId":"","CallerIp":"","AppId":0,"CalleeAction":"","InstanceId":"","Action":"","CostTime":0,"Callee":"","FunctionType":"","LogContent":"pull image fail: native export failed to resolve image \"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2\" (if this is a multi-arch index, verify the requested platform exists): Get \"https://harbor-intranet.allintechinc.com/v2/\": dial tcp: lookup harbor-intranet.allintechinc.com: no such host","CodeLine":"","@timestamp":"2026-09-20T17:28:33.837265159+08:00","job_id":"00306852-77df-4edd-895e-d302a7dbad29","RetCode":0,"Version":"v0.7.1","image":"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2","template_id":"tpl-7a670f919698434eaeb8a15d","LocalIp":"192.168.0.6","CalleeEndpoint":"","Namespace":""} {"CostTime":0,"RequestId":"","CallerIp":"","Callee":"","Namespace":"","LogContent":"report status to master: job_id=00306852-77df-4edd-895e-d302a7dbad29 url=http://127.0.0.1:8089/internal/template/jobs/00306852-77df-4edd-895e-d302a7dbad29/status fields=map[error_message:pull image fail: native export failed to resolve image \"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2\" (if this is a multi-arch index, verify the requested platform exists): Get \"https://harbor-intranet.allintechinc.com/v2/\": dial tcp: lookup harbor-intranet.allintechinc.com: no such host phase:PULLING progress:100 status:FAILED]","LocalIp":"192.168.0.6","LogLevel":"INFO","InstanceType":"","Action":"","ContainerId":"","InstanceId":"","Module":"templatecenter","CodeLine":"","@timestamp":"2026-09-20T17:28:33.837335442+08:00","CalleeEndpoint":"","RetCode":0,"CalleeAction":"","AppId":0,"Caller":"","Version":"v0.7.1","FunctionType":""} {"Callee":"","CalleeEndpoint":"","RequestId":"","FunctionType":"","InstanceId":"","InstanceType":"","LocalIp":"192.168.0.6","@timestamp":"2026-09-20T17:28:33.841149682+08:00","LogLevel":"ERROR","CallerIp":"","CalleeAction":"","CostTime":0,"Namespace":"","ContainerId":"","RetCode":0,"Version":"v0.7.1","Caller":"","LogContent":"build template fail: job_id=00306852-77df-4edd-895e-d302a7dbad29 err=pull image: native export failed to resolve image \"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2\" (if this is a multi-arch index, verify the requested platform exists): Get \"https://harbor-intranet.allintechinc.com/v2/\": dial tcp: lookup harbor-intranet.allintechinc.com: no such host","Module":"templatecenter","AppId":0,"Action":"","CodeLine":""} {"LocalIp":"192.168.0.6","CodeLine":"","CalleeEndpoint":"","Namespace":"","AppId":0,"RequestId":"","LogContent":"report status attempt 1/6 failed (retrying in 500ms): job_id=d81ce25e-24d9-463c-87d3-6d7e554e5ba6 err=master returned 500: {\"error\":\"record not found\"}","LogLevel":"WARN","InstanceType":"","Version":"v0.7.1","FunctionType":"","CostTime":0,"Callee":"","Caller":"","RetCode":0,"CalleeAction":"","Module":"templatecenter","@timestamp":"2026-09-20T17:37:20.440439577+08:00","ContainerId":"","CallerIp":"","InstanceId":"","Action":""}
从日志看，CubeTemplateCenter 本身已经恢复正常启动，http_bind=192.168.0.6 修改不是当前故障点
现在真正失败的是 模板构建阶段拉取 Harbor 镜像失败：
pull image fail:
native export failed to resolve image
"harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2"
invalid realm in www-authenticate:
realm scheme "http" not allowed for a secure registry; use https
之前还有：
dial tcp: lookup harbor-intranet.allintechinc.com: no such host
说明问题经历了两个阶段：
- DNS 不通 ✅ 已经修复
- Harbor HTTPS / Registry Auth 配置异常 ❌ 当前问题
1. 当前调用链分析
现在流程：
CubeTemplateCenter
|
|
v
containerd / nerdctl
|
|
v
https://harbor-intranet.allintechinc.com/v2/
|
|
v
Harbor Registry
|
|
X
WWW-Authenticate 返回 realm=http://...
关键：
Harbor 对外访问是 HTTPS：
https://harbor-intranet.allintechinc.com
但是 Harbor 返回认证地址：
WWW-Authenticate:
Bearer realm="http://harbor-intranet.allintechinc.com/service/token"
containerd 认为：
一个 HTTPS Registry，认证跳转却是 HTTP，不安全，拒绝
2. 检查 Harbor 配置
登录 Harbor 节点
检查：
grep hostname /harbor/harbor.yml
应该类似：
hostname: harbor-intranet.allintechinc.com
检查：
grep external_url /harbor/harbor.yml
如果存在：
应该：
external_url: https://harbor-intranet.allintechinc.com
不要：
external_url: http://harbor-intranet.allintechinc.com
3. 检查 nginx / proxy 配置
Harbor 默认：
nginx
|
|
core
|
|
registry
查看：
docker ps | grep nginx
进入：
docker exec -it nginx bash
查看：
grep -R "proxy_set_header X-Forwarded" /etc/nginx/
必须有：
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header Host $host;
重点：
如果缺：
X-Forwarded-Proto https
Harbor Core 会认为自己是 HTTP
于是生成：
realm=http://...
4. 修改 Harbor nginx
找到：
harbor/common/config/nginx/nginx.conf
或者：
/data/harbor/common/config/nginx/nginx.conf
检查：
location /service/ {
proxy_pass http://core;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto https;
}
补充：
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port 443;
5. 重新生成 Harbor 配置
Harbor目录：
例如：
cd /opt/harbor
执行：
./prepare
重启：
docker compose down
docker compose up -d
确认：
docker ps
6. 验证 Harbor API
在 CubeTemplateCenter 机器：
curl -vk https://harbor-intranet.allintechinc.com/v2/
正常：
可能返回：
HTTP/1.1 401 Unauthorized
WWW-Authenticate:
Bearer realm="https://harbor-intranet.allintechinc.com/service/token"
注意：
必须是：
realm="https://..."
不能：
realm="http://..."
7. containerd 测试
CubeTemplateCenter 使用 native export，大概率走 containerd
检查：
crictl info | grep runtime
测试拉取：
ctr images pull \
harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2
或者：
nerdctl pull \
harbor-intranet.allintechinc.com/llm-planning/llm-planning-sandbox:v0.0.2
如果私有仓库：
nerdctl login harbor-intranet.allintechinc.com
8. 如果 Harbor 是自签 HTTPS
还需要给 containerd 加 CA
检查：
openssl s_client \
-connect harbor-intranet.allintechinc.com:443 \
-showcerts
安装：
mkdir -p /etc/containerd/certs.d/harbor-intranet.allintechinc.com
cp harbor-ca.crt \
/etc/containerd/certs.d/harbor-intranet.allintechinc.com/ca.crt
配置：
cat >/etc/containerd/certs.d/harbor-intranet.allintechinc.com/hosts.toml <<EOF
server = "https://harbor-intranet.allintechinc.com"
[host."https://harbor-intranet.allintechinc.com"]
capabilities = ["pull", "resolve"]
ca = "/etc/containerd/certs.d/harbor-intranet.allintechinc.com/ca.crt"
EOF
重启：
systemctl restart containerd
9. 关于 templatecenter 的 master 500
这个：
master returned 500:
{"error":"record not found"}
不是根因
原因：
templatecenter 创建任务
|
|
pull image失败
|
|
状态回报失败
|
|
master找不到job记录
真正根因：
Harbor registry authentication
当前状态总结
| 组件 | 状态 |
|---|---|
| CubeTemplateCenter启动 | ✅正常 |
| http_bind=192.168.0.6 | ✅正常 |
| 8090监听 | ✅正常 |
| DNS harbor-intranet | ✅已解决 |
| Harbor认证 | ❌异常 |
| containerd拉镜像 | ❌失败 |
转载自 CSDN-专业IT技术社区
原文链接：https://blog.csdn.net/qq_38636133/article/details/166138955
