完善了对protobuf的支持

CODE_LOGIC_OVERVIEW.md

@@ -0,0 +1,288 @@
+# EFKA 当前代码逻辑梳理
+
+> 本文档记录当前阅读代码后对项目逻辑的理解，主要基于 `apps/efka/src/`、`apps/efka/include/`、`config/` 和 protobuf 生成文件。尚未覆盖每个边界条件和全部测试行为。
+
+## 1. 项目定位
+
+这个项目是一个 Erlang/OTP 应用，核心作用像一个边缘侧 agent：
+
+- 对本机微服务开放 WebSocket 接入，微服务注册、订阅 topic、上报指标。
+- 与上游服务器建立 TLS 长连接，完成鉴权、上报数据、接收远程命令。
+- 通过 Docker API 管理本机容器，包括部署、启动、停止、杀掉、删除、更新配置和列表查询。
+- 用 DETS 做少量本地持久化，包括服务状态和离线缓存包。
+
+依赖主要有：
+
+- `cowboy`：本地 WebSocket server。
+- `gun`：通过 Unix Socket 访问 Docker HTTP API。
+- `ssl`：连接上游 TLS server。
+- OTP `json` 模块：JSON 编解码。
+- `gpb`：protobuf 代码生成。
+
+## 2. 启动流程
+
+应用入口是 `efka_app:start/2`。
+
+启动步骤：
+
+1. 设置 Erlang IO 编码为 unicode。
+2. 设置 `fullsweep_after`，试图加快进程内存回收。
+3. 启动 Cowboy WebSocket server。
+4. 启动顶层 supervisor `efka_sup`。
+
+WebSocket server：
+
+- 从 `config/sys.config` 读取 `efka.websocket_server`。
+- 监听 `/ws`。
+- 请求处理模块是 `efka_service_channel`。
+
+`efka_sup` 启动的子进程：
+
+- `efka_logger`：部署日志落盘。
+- `efka_service_sup`：动态管理每个已注册微服务对应的 `efka_service` 进程。
+- `efka_service_model`：DETS 服务状态表。
+- `efka_subscription`：本地 topic 订阅中心。
+- `efka_iot_client`：连接上游 TLS server 的状态机。
+
+`docker` application 启动的子进程：
+
+- `docker_task_reporter`：部署任务事件缓冲和转发。
+- `docker_deploy_manager`：部署任务管理器。
+
+`docker_events` 代码存在于 `apps/docker`，但目前默认不会启动。
+
+## 3. 两条通信链路
+
+项目里有两条主要通信链路。
+
+### 3.1 微服务到 EFKA：WebSocket
+
+本机微服务连接 `ws://<host>:18080/ws`。
+
+协议处理在 `efka_service_channel`：
+
+- WebSocket 收到 `ping` 直接回复 `pong`。
+- 收到 binary 帧时，第一个字节是帧类型。
+- `FRAME_REQUEST = 0x01`，用 `efka_service_pb` 解码为 `ServiceRequest`。
+- `FRAME_CAST = 0x03`，用 `efka_service_pb` 解码为 `ServiceCast`。
+- 回复时使用 `FRAME_REPLY = 0x02`。
+
+支持的微服务请求：
+
+- `register`：注册服务。
+- `subscribe`：订阅 topic。
+
+支持的微服务 cast：
+
+- `metric_data`：上报指标数据。
+
+注册流程：
+
+1. 微服务发送 `ServiceRequest.Register`，包含 `service_id`。
+2. `efka_service_channel` 调用 `efka_service_sup:start_service(ServiceId)`。
+3. 如果该服务进程不存在，则动态启动一个 `efka_service`；如果已存在，复用已有 pid。
+4. `efka_service_channel` 调用 `efka_service:attach_channel(ServicePid, self())` 绑定当前 WebSocket channel。
+5. `efka_service` 只允许绑定一个存活 channel；已有 channel 存活时返回 `channel exists`。
+6. 注册成功后写入 `efka_service_model`，服务状态设为 running。
+7. channel 退出时，取消全部订阅，并把服务状态改为 stopped。
+
+指标上报流程：
+
+1. 微服务发送 `ServiceCast.MetricData`。
+2. `efka_service_channel` 调用 `efka_service:metric_data(ServicePid, RouteKey, Metric)`。
+3. `efka_service` 转发给 `efka_iot_client:metric_data(RouteKey, Metric)`。
+4. `efka_iot_client` 如果处于 activated 状态，直接发给上游；否则写入持久化 outbox。
+
+### 3.2 EFKA 到上游：TLS 长连接
+
+上游连接逻辑在 `efka_iot_client`，它是一个 `gen_statem`。
+
+状态包括：
+
+- `disconnected`：未连接。
+- `auth`：已连接，等待鉴权响应。
+- `restricted`：鉴权受限，不能正常推送数据，但可以接收部分授权命令。
+- `activated`：已激活，可正常收发。
+
+连接流程：
+
+1. 初始化后立即触发 `create_transport`。
+2. 读取 `efka.tls_server_address`，用 `ssl:connect/4` 建立 TLS 连接。
+3. TLS socket 使用 `{packet, 4}` 和 `{active, true}`。
+4. 连接成功后发送 `AuthRequest`。
+5. 收到鉴权成功 reply 后进入 `activated`，并触发 outbox 刷出。
+6. 连接或鉴权失败时关闭 socket，5 秒后重连。
+
+上游协议同样用第一个字节区分帧类型：
+
+- `FRAME_REQUEST = 0x01`
+- `FRAME_REPLY = 0x02`
+- `FRAME_CAST = 0x03`
+
+但 protobuf 使用的是 `message_pb`，不是微服务侧的 `efka_service_pb`。
+
+`efka_iot_client` 上报的内容：
+
+- `metric_data`：业务指标数据，activated 时实时发送，否则进入持久化 outbox。
+- `task_event_stream`：Docker 部署任务流式日志，只在 activated 时发送。
+- `close_task_event_stream`：任务结束事件，只在 activated 时发送。
+
+`efka_iot_client` 接收的内容：
+
+- `RequestFrame.container_request`：远程容器管理请求，交给 `docker_container_service`。
+- `CastFrame.command`：授权命令，`COMMAND_AUTH` 用于切换鉴权/受限状态。
+- `CastFrame.pub`：上游发布的 topic 消息，交给 `efka_subscription:publish/3`。
+
+## 4. 本地 Pub/Sub 逻辑
+
+`efka_subscription` 是本地订阅中心。
+
+订阅：
+
+- `efka_service_channel` 收到微服务 `subscribe` 后调用 `efka_subscription:subscribe(Topic, self())`。
+- 同一个 pid 对同一个 topic 重复订阅会被忽略。
+- 首次看到某个 subscriber pid 时，会 monitor 该 pid；pid 退出后自动移除订阅。
+
+topic 匹配规则：
+
+- `/` 分隔 topic components。
+- `*` 表示单级匹配。
+- `+` 表示多级匹配，但只能出现在末尾。
+- 完全匹配优先级最高，不过当前代码里的 `order` 字段只是计算并保存，没有实际排序使用。
+
+发布：
+
+- 上游发来的 `CastFrame.Pub` 会调用 `efka_subscription:publish(Topic, Qos, Content)`。
+- 如果匹配到订阅者，向对应 `efka_service_channel` 发送 `{topic_broadcast, Topic, Content}`。
+- `efka_service_channel` 再编码为 `ServiceCast.TopicEvent` 推送给微服务。
+- 如果没有订阅者且 `Qos != 0`，消息会暂存在 `remaining_messages`。
+- 新订阅建立时，会尝试把匹配的 remaining messages 补发给该订阅者。
+
+## 5. Docker 容器管理链路
+
+上游发来的容器请求最终进入 `docker_container_service:handle_request/1`。
+
+支持动作：
+
+- `list`：列出容器。
+- `deploy`：部署容器。
+- `start`：启动容器。
+- `stop`：停止容器。
+- `kill`：发送 kill 信号。
+- `remove`：删除容器。
+- `config`：更新容器配置文件。
+
+简单操作：
+
+- `start/stop/kill/remove/list` 直接调用 `docker_commands`。
+- `docker_commands` 通过短生命周期普通 `docker_client` 进程访问 `/var/run/docker.sock`。
+- HTTP 客户端用 `gun:open_unix/2`。
+- 流式请求使用 `{docker_client, Ref, Event}` 消息返回，事件包括 `{response, Status, Headers}`、`{data, Bin}`、`done` 和 `{error, Reason}`。
+
+部署操作：
+
+1. `docker_container_service` 收到 `deploy` 后调用 `docker_deploy_manager:deploy(TaskId, Params)`。
+2. `docker_deploy_manager` 根据 `docker.root_dir` 和参数里的 `container_name/container_dir` 确保容器目录存在。
+3. 它启动一个独立 `docker_deployer` 进程，并 monitor 该部署进程。
+4. `docker_deployer` 执行实际部署步骤。
+5. 部署过程事件写入 `docker_task_reporter`。
+6. `docker_task_reporter` 在上游连接 activated 时把事件转给 `efka_iot_client`；未激活时保留队列并定时重试。
+
+`docker_deployer` 当前部署步骤：
+
+1. 上报“开始部署容器”。
+2. 调用 `ensure_container_absent/2`，但当前实现只是上报“开始创建容器”，没有真正删除旧容器。
+3. 规范化镜像名，没有 tag 时补 `:latest`。
+4. 调用 Docker API 拉取镜像，`docker_deployer` 按 `Ref` 接收 `docker_client` 流式消息并上报拉取过程。
+5. 创建容器。
+6. 创建空的 `service.conf` 配置文件。
+7. 写部署摘要到 `efka_logger`。
+8. 关闭任务事件流，状态为 `success` 或 `fail`。
+
+创建容器参数：
+
+- `docker_container_builder` 把 protobuf 的 `DockerCreateOptions` 转成 Docker API JSON。
+- 会自动给容器注入环境变量 `CONTAINER_NAME=<name>`。
+- 会自动添加容器内配置路径 `/usr/local/etc/service.conf`。
+- 会自动把宿主机 `service.conf` bind 到容器内 `/usr/local/etc/service.conf`。
+
+配置更新：
+
+- `docker_container_service:update_container_config/2` 根据容器名找到目录。
+- 写入该目录下的 `service.conf`。
+- 容器目录映射由 `docker_helper` 通过 `.container_dir` 指针文件维护。
+
+## 6. 本地持久化
+
+### 6.1 `efka_service_model`
+
+使用 DETS 表 `service`。
+
+文件位置来自 `efka.dets_dir`，默认配置是：
+
+```erlang
+"/usr/local/code/tmp/dets/service.dets"
+```
+
+记录结构在 `apps/efka/include/efka_tables.hrl`：
+
+- `service_id`
+- `container_name`
+- `meta_data`
+- `status`
+- `create_ts`
+- `update_ts`
+
+用途：
+
+- 注册成功时插入或更新服务。
+- channel 关闭时把服务状态改成 stopped。
+- 支持查询所有服务、运行中服务和单个服务状态。
+
+### 6.2 `efka_iot_outbox`
+
+使用 append-only log 文件和 metadata 文件。
+
+作用：
+
+- 当 `efka_iot_client` 不在 activated 状态时，把待上报 packet 缓存下来。
+- `efka_iot_client` 激活后循环 `next -> send -> ack` 刷出。
+- 所有记录 ack 后会截断 log，下一轮从 seq 1 重新开始。
+
+## 7. 日志
+
+有两套日志：
+
+- OTP logger：在 `config/sys.config` 里配置到 console 和 `log/debug.log`。
+- `efka_logger`：自定义部署日志，按日期写到 `code:root_dir() ++ "/log/"` 下。
+
+`efka_logger` 主要用于部署任务摘要和失败原因。
+
+## 8. 配置项
+
+关键配置来自 `config/sys.config`：
+
+- `docker.root_dir`：容器相关目录根路径。
+- `dets_dir`：DETS 文件目录。
+- `websocket_server`：本地 WebSocket 监听配置。
+- `tls_server_address`：上游 TLS server 地址。
+- `auth`：上游鉴权信息。
+
+注意：
+
+- `efka_service_model` 打开 DETS 前假设 `dets_dir` 已存在，代码里没有显式创建目录。
+- `docker_client` 固定使用 `/var/run/docker.sock`，每次请求内部由短生命周期普通进程执行 open/request/close。
+
+## 9. 当前看到的几个注意点
+
+- `README.md` 描述的是 JSON-RPC 风格 WebSocket，但当前代码实际处理的是 binary protobuf 帧；README 可能已经过期。
+- `efka_service_channel` 里 `register` 只使用 `service_id`，没有使用 README 中提到的 `meta_data/container_name`。
+- `efka_iot_client:send_result_reply/3` 和 `send_error_reply/3` 编码 `ReplyFrame` 后没有加 `FRAME_REPLY` 前缀；接收侧是否期望裸 protobuf 需要确认。
+- `docker_deployer:ensure_container_absent/2` 当前没有真正确保旧容器不存在，只是上报日志。
+- `efka_subscription` 计算了 topic `order`，但匹配广播时没有使用优先级排序。
+- `docker_commands` 部分错误响应解码没有统一使用 `[return_maps]`，有些分支可能匹配不到 map。
+- `docker_events` 存在但未启动，且使用 shell 命令 `docker events`，与其他 Docker API 访问方式不同。
+
+## 10. 一句话主流程
+
+微服务通过 WebSocket 注册到 EFKA，本地 channel 把指标交给对应 `efka_service`，再由 `efka_iot_client` 通过 TLS 上报给上游；上游通过同一条 TLS 连接下发 pub/sub 消息和容器管理请求，pub/sub 再广播回本地微服务，容器请求则通过 Docker Unix Socket 在本机执行。

README.md

@@ -1,13 +1,110 @@
-efka
-=====
+# ws_channel 模块 API 文档与交互逻辑
 
-An OTP application
+## 注意websocket的数据格式为: text
 
-1. 先解决数据的上行问题
-2. todo list
-    要解决连接断开重新连接的问题 !!!
+## 一、模块概述
+`ws_channel` 是基于 Erlang + Cowboy WebSocket 实现的 MQTT 相关交互模块，主要用于服务注册、主题订阅、指标数据上报、事件发送及消息广播等功能，通过 WebSocket 协议实现客户端与服务端的实时双向通信。
 
-Build
------
 
-    $ rebar3 compile
+## 三、核心 API 方法
+客户端通过发送 **JSON 格式文本消息** 与服务端交互，消息格式遵循 JSON-RPC 风格（包含 `id`、`method`、`params` 字段）。
+
+
+### 1. 服务注册（register）
+#### 功能
+注册服务并建立客户端与服务进程的关联，是后续操作（订阅、上报数据等）的前提。
+
+#### 请求格式
+```json
+{
+  "id": <整数，请求唯一标识>,
+  "method": "register",
+  "params": {
+    "service_id": <二进制，服务唯一标识，必填>,
+    "meta_data": <映射，服务元数据，可选>,
+    "container_name": <二进制，容器名称，可选>
+  }
+}
+```
+
+#### 响应格式
+- 成功响应：
+  ```json
+  {
+    "id": <与请求id一致>,
+    "result": "ok"
+  }
+  ```
+- 失败处理：服务端直接关闭连接（因 `attach_channel` 失败）
+
+### 2. 主题订阅（subscribe）
+#### 功能
+订阅指定主题，后续可接收该主题的广播消息。
+
+#### 请求格式
+```json
+{
+  "id": <整数，请求唯一标识>,
+  "method": "subscribe",
+  "params": {
+    "topic": <二进制，订阅的主题名称，必填>
+  }
+}
+```
+
+#### 响应格式
+- 成功响应：
+  ```json
+  {
+    "id": <与请求id一致>,
+    "result": "ok"
+  }
+  ```
+- 失败响应：
+  ```json
+  {
+    "id": <与请求id一致>,
+    "error": {
+      "code": -1,
+      "message": "错误描述"
+    }
+  }
+  ```
+
+#### 处理逻辑
+通过 `efka_subscription:subscribe(Topic, self())` 完成订阅，订阅成功后客户端会收到该主题的广播消息。
+
+
+### 3. 指标数据上报（metric_data）
+#### 功能
+向服务进程上报设备指标数据。
+
+#### 请求格式
+```json
+{
+  "method": "metric_data",
+  "params": {
+    "route_key": <二进制，路由键，必填>,
+    "metric": <指标数据，必填>
+  }
+}
+```
+
+#### 响应处理
+服务端接收后无返回消息（处理逻辑：`efka_service:metric_data(ServicePid, DeviceUUID, RouteKey, Metric)`）
+
+## 五、基础交互协议
+1. **Ping/Pong 心跳**：
+    - 客户端发送 `ping` 消息
+    - 服务端回复 `pong` 消息，维持连接
+
+2. **未知消息处理**：
+    - 客户端发送未定义格式的消息时，服务端记录错误并关闭连接
+
+## 七、典型交互流程
+1. 客户端发起 WebSocket 连接
+2. 客户端发送 `register` 请求完成注册
+3. 客户端发送 `subscribe` 请求订阅目标主题
+4. 客户端通过 `metric_data` 上报指标数据 / 通过 `event` 发送事件
+5. 服务端向客户端推送已订阅主题的消息（`publish` 方法）
+6. 连接关闭（主动断开或异常终止）

apps/docker/src/docker.app.src

@@ -0,0 +1,18 @@
+{application, docker,
+ [{description, "Docker integration application"},
+  {vsn, "0.1.0"},
+  {registered, []},
+  {mod, {docker_app, []}},
+  {applications,
+   [
+    gun,
+    kernel,
+    stdlib
+   ]},
+  {env, [
+    {root_dir, "/usr/local/code/efka"}
+  ]},
+  {modules, []},
+  {licenses, ["Apache-2.0"]},
+  {links, []}
+ ]}.

apps/docker/src/docker_app.erl

@@ -0,0 +1,18 @@
+%%%-------------------------------------------------------------------
+%% @doc Docker application entry point.
+%% @end
+%%%-------------------------------------------------------------------
+
+-module(docker_app).
+
+-behaviour(application).
+
+-export([start/2, stop/1]).
+
+-spec start(term(), term()) -> {ok, pid()} | {error, term()}.
+start(_StartType, _StartArgs) ->
+    docker_sup:start_link().
+
+-spec stop(term()) -> ok.
+stop(_State) ->
+    ok.

apps/docker/src/docker_client.erl

@@ -0,0 +1,292 @@
+%%%-------------------------------------------------------------------
+%%% @doc
+%%% Short-lived Docker API client process.
+%%%
+%%% Each request owns one process and one gun Unix socket connection.
+%%% The process exits after the Docker API response finishes.
+%%% @end
+%%%-------------------------------------------------------------------
+-module(docker_client).
+
+%% API
+-export([request/4, start_stream/5, start_stream_no_timeout/5]).
+
+-define(DOCKER_SOCKET, "/var/run/docker.sock").
+-define(RESPONSE_TIMEOUT, 5000).
+-define(BODY_TIMEOUT, 10000).
+-define(STREAM_BODY_TIMEOUT, 30000).
+-define(CLIENT_TIMEOUT, 60000).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec request(Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
+    {ok, StatusCode :: integer(), RespHeaders :: proplists:proplist(), RespBody :: binary()} | {error, any()}.
+request(Method, Path, Body, Headers) when is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
+    Owner = self(),
+    Ref = make_ref(),
+    {Pid, MRef} = spawn_monitor(fun() -> request_worker(Owner, Ref, Method, Path, Body, Headers) end),
+    receive
+        {docker_client, Ref, {result, Result}} ->
+            erlang:demonitor(MRef, [flush]),
+            Result;
+        {'DOWN', MRef, process, Pid, Reason} ->
+            {error, {client_down, Reason}}
+    after ?CLIENT_TIMEOUT ->
+        exit(Pid, shutdown),
+        erlang:demonitor(MRef, [flush]),
+        {error, timeout}
+    end.
+
+-spec start_stream(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
+    {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}.
+start_stream(Owner, Method, Path, Body, Headers)
+  when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
+    Ref = make_ref(),
+    {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, ?RESPONSE_TIMEOUT, ?STREAM_BODY_TIMEOUT) end),
+    {ok, Ref, Pid, MRef}.
+
+-spec start_stream_no_timeout(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
+    {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}.
+start_stream_no_timeout(Owner, Method, Path, Body, Headers)
+  when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
+    Ref = make_ref(),
+    {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, infinity, infinity) end),
+    {ok, Ref, Pid, MRef}.
+
+%%%===================================================================
+%%% Worker functions
+%%%===================================================================
+
+-spec request_worker(pid(), reference(), string(), string(), binary(), list()) -> ok.
+request_worker(Owner, Ref, Method, Path, Body, Headers) ->
+    Result = do_request(Method, Path, Body, Headers),
+    send_owner(Owner, Ref, {result, Result}).
+
+-spec stream_worker(pid(), reference(), string(), string(), binary(), list(), timeout(), timeout()) -> ok.
+stream_worker(Owner, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) ->
+    OwnerRef = erlang:monitor(process, Owner),
+    _Result = do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout),
+    erlang:demonitor(OwnerRef, [flush]),
+    ok.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec do_request(string(), string(), binary(), list()) ->
+    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
+do_request(Method, Path, Body, Headers) ->
+    case open_connection() of
+        {ok, ConnPid} ->
+            try
+                StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
+                receive_response(ConnPid, StreamRef)
+            after
+                close_connection(ConnPid)
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec do_stream(pid(), reference(), reference(), string(), string(), binary(), list(), timeout(), timeout()) ->
+    ok | {error, binary()}.
+do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) ->
+    case open_connection() of
+        {ok, ConnPid} ->
+            try
+                StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
+                receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout)
+            after
+                close_connection(ConnPid)
+            end;
+        {error, Reason0} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason}
+    end.
+
+-spec receive_response(pid(), reference()) ->
+    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
+receive_response(ConnPid, StreamRef) ->
+    receive
+        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} ->
+            receive_body(ConnPid, StreamRef, Status, Headers, <<>>);
+        {gun_response, ConnPid, StreamRef, fin, Status, Headers} ->
+            {ok, Status, Headers, <<>>};
+        {gun_error, ConnPid, StreamRef, Reason} ->
+            {error, {http_error, Reason}};
+        {gun_error, ConnPid, Reason} ->
+            {error, {http_error, Reason}};
+        {gun_down, ConnPid, _, Reason, _} ->
+            {error, {http_closed, Reason}}
+    after ?RESPONSE_TIMEOUT ->
+        {error, timeout}
+    end.
+
+-spec receive_body(pid(), reference(), integer(), proplists:proplist(), iodata()) ->
+    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
+receive_body(ConnPid, StreamRef, Status, Headers, Acc) ->
+    receive
+        {gun_data, ConnPid, StreamRef, fin, Data} ->
+            Body = iolist_to_binary([Acc, Data]),
+            {ok, Status, Headers, Body};
+        {gun_data, ConnPid, StreamRef, nofin, Data} ->
+            receive_body(ConnPid, StreamRef, Status, Headers, [Acc, Data]);
+        {gun_error, ConnPid, StreamRef, Reason} ->
+            {error, {http_error, Reason}};
+        {gun_error, ConnPid, Reason} ->
+            {error, {http_error, Reason}};
+        {gun_down, ConnPid, _, Reason, _} ->
+            {error, {http_closed, Reason}}
+    after ?BODY_TIMEOUT ->
+        {error, timeout}
+    end.
+
+-spec receive_stream_response(pid(), reference(), reference(), pid(), reference(), timeout(), timeout()) -> ok | {error, binary()}.
+receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout) ->
+    receive
+        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} when Status >= 200, Status < 300 ->
+            send_owner(Owner, Ref, {response, Status, Headers}),
+            receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout);
+        {gun_response, ConnPid, StreamRef, fin, Status, Headers} when Status >= 200, Status < 300 ->
+            send_owner(Owner, Ref, {response, Status, Headers}),
+            send_owner(Owner, Ref, done),
+            ok;
+        {gun_response, ConnPid, StreamRef, nofin, Status, _Headers} ->
+            receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, <<>>, stream_error_body_timeout(BodyTimeout));
+        {gun_response, ConnPid, StreamRef, fin, Status, _Headers} ->
+            Reason = http_status_error(Status, <<>>),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_error, ConnPid, StreamRef, Reason0} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_error, ConnPid, Reason0} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_down, ConnPid, _, Reason0, _} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
+            {error, format_error({owner_down, Reason0})}
+    after ResponseTimeout ->
+        send_owner(Owner, Ref, {error, <<"处理超时"/utf8>>}),
+        {error, <<"timeout">>}
+    end.
+
+-spec receive_stream_body(pid(), reference(), reference(), pid(), reference(), timeout()) -> ok | {error, binary()}.
+receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout) ->
+    receive
+        {gun_data, ConnPid, StreamRef, fin, Data} ->
+            maybe_send_stream_data(Owner, Ref, Data),
+            send_owner(Owner, Ref, done),
+            ok;
+        {gun_data, ConnPid, StreamRef, nofin, Data} ->
+            maybe_send_stream_data(Owner, Ref, Data),
+            receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout);
+        {gun_error, ConnPid, StreamRef, Reason0} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_error, ConnPid, Reason0} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_down, ConnPid, _, Reason0, _} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
+            {error, format_error({owner_down, Reason0})}
+    after BodyTimeout ->
+        send_owner(Owner, Ref, {error, <<"timeout">>}),
+        {error, <<"timeout">>}
+    end.
+
+-spec receive_stream_error_body(pid(), reference(), reference(), pid(), reference(), integer(), iodata(), timeout()) ->
+    {error, binary()}.
+receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, Acc, BodyTimeout) ->
+    receive
+        {gun_data, ConnPid, StreamRef, fin, Data} ->
+            Reason = http_status_error(Status, iolist_to_binary([Acc, Data])),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_data, ConnPid, StreamRef, nofin, Data} ->
+            receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, [Acc, Data], BodyTimeout);
+        {gun_error, ConnPid, StreamRef, Reason0} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_error, ConnPid, Reason0} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {gun_down, ConnPid, _, Reason0, _} ->
+            Reason = format_error(Reason0),
+            send_owner(Owner, Ref, {error, Reason}),
+            {error, Reason};
+        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
+            {error, format_error({owner_down, Reason0})}
+    after BodyTimeout ->
+        Reason = http_status_error(Status, iolist_to_binary(Acc)),
+        send_owner(Owner, Ref, {error, Reason}),
+        {error, Reason}
+    end.
+
+-spec stream_error_body_timeout(timeout()) -> timeout().
+stream_error_body_timeout(infinity) ->
+    infinity;
+stream_error_body_timeout(_BodyTimeout) ->
+    ?BODY_TIMEOUT.
+
+-spec open_connection() -> {ok, pid()} | {error, any()}.
+open_connection() ->
+    case gun:open_unix(?DOCKER_SOCKET, #{}) of
+        {ok, ConnPid} ->
+            case gun:await_up(ConnPid) of
+                {ok, _} ->
+                    {ok, ConnPid};
+                {error, Reason} ->
+                    close_connection(ConnPid),
+                    {error, Reason}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec close_connection(pid()) -> ok.
+close_connection(ConnPid) ->
+    catch gun:close(ConnPid),
+    ok.
+
+-spec send_owner(pid(), reference(), term()) -> ok.
+send_owner(Owner, Ref, Event) ->
+    Owner ! {docker_client, Ref, Event},
+    ok.
+
+-spec maybe_send_stream_data(pid(), reference(), iodata()) -> ok.
+maybe_send_stream_data(Owner, Ref, Data) ->
+    case iolist_to_binary(Data) of
+        <<>> ->
+            ok;
+        DataBin ->
+            send_owner(Owner, Ref, {data, DataBin})
+    end.
+
+-spec http_status_error(integer(), binary()) -> binary().
+http_status_error(Status, <<>>) ->
+    iolist_to_binary(io_lib:format("docker http status ~B", [Status]));
+http_status_error(Status, Body) when is_binary(Body) ->
+    StatusBin = integer_to_binary(Status),
+    <<"docker http status ", StatusBin/binary, ": ", Body/binary>>.
+
+-spec format_error(term()) -> binary().
+format_error(Reason) when is_binary(Reason) ->
+    Reason;
+format_error(Reason) ->
+    iolist_to_binary(io_lib:format("~p", [Reason])).

apps/docker/src/docker_commands.erl

@@ -0,0 +1,295 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 15. 9月 2025 16:11
+%%%-------------------------------------------------------------------
+-module(docker_commands).
+-author("anlicheng").
+
+%% API
+-export([pull_image/1, check_image_exist/1]).
+-export([create_container/2, check_container_exist/1, is_container_running/1,
+    start_container/1, stop_container/1, stop_container/2, remove_container/1, remove_container/3, kill_container/1, kill_container/2,
+    get_containers/0]).
+
+-spec pull_image(Image :: binary()) -> {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()} | {error, Reason :: any()}.
+pull_image(Image) when is_binary(Image) ->
+    Url = lists:flatten(io_lib:format("/images/create?fromImage=~s", [binary_to_list(Image)])),
+    docker_client:start_stream_no_timeout(self(), "POST", Url, <<>>, []).
+
+-spec check_image_exist(Image :: binary()) -> boolean().
+check_image_exist(Image) when is_binary(Image) ->
+    EncodedImage = uri_string:quote(Image),
+    Url = lists:flatten(io_lib:format("/images/~s/json", [binary_to_list(EncodedImage)])),
+    case docker_client:request("GET", Url, <<"">>, []) of
+        {ok, 200, _Headers, _Resp} ->
+            true;
+        {ok, 404, _, _} ->
+            false;
+        _ ->
+            false
+    end.
+
+-spec create_container(ContainerDir :: string(), Params :: map()) ->
+    {ok, ContainerId :: binary()} | {error, Reason :: any()}.
+create_container(ContainerDir, #{<<"container_name">> := ContainerName0, <<"create">> := CreateOpts})
+    when is_list(ContainerDir), is_binary(ContainerName0), is_map(CreateOpts) ->
+    Url = lists:flatten(io_lib:format("/containers/create?name=~s", [binary_to_list(ContainerName0)])),
+    Options = docker_container_builder:build_options(ContainerName0, ContainerDir, CreateOpts),
+    display_options(Options),
+
+    Body = iolist_to_binary(json:encode(Options)),
+
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+
+    case docker_client:request("POST", Url, Body, Headers) of
+        {ok, 201, _Headers, Resp} ->
+            case catch json:decode(Resp) of
+                #{<<"Id">> := ContainerId} when is_binary(ContainerId) ->
+                    {ok, ContainerId};
+                _ ->
+                    {error, Resp}
+            end;
+        {ok, _StatusCode, _, ErrorResp} ->
+            case catch json:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end;
+create_container(_ContainerDir, _Params) ->
+    {error, <<"invalid container params">>}.
+
+-spec is_container_running(ContainerId :: binary()) -> boolean().
+is_container_running(ContainerId) when is_binary(ContainerId) ->
+    case inspect_container(ContainerId) of
+        {ok, #{<<"State">> := #{<<"Running">> := Running}}} ->
+            Running;
+        {error, _} ->
+            false
+    end.
+
+-spec check_container_exist(ContainerName :: binary()) -> boolean().
+check_container_exist(ContainerName) when is_binary(ContainerName) ->
+    case inspect_container(ContainerName) of
+        {ok, #{<<"Id">> := Id}} when is_binary(Id) ->
+            true;
+        _ ->
+            false
+    end.
+
+-spec start_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+start_container(ContainerName) when is_binary(ContainerName) ->
+    Url = lists:flatten(io_lib:format("/containers/~s/start", [binary_to_list(ContainerName)])),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_client:request("POST", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, 304, _Headers, _} ->
+            {error, <<"container already started">>};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch json:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec stop_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+stop_container(ContainerName) when is_binary(ContainerName) ->
+    stop_container(ContainerName, 0).
+
+-spec stop_container(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> ok | {error, Reason :: binary()}.
+stop_container(ContainerName, TimeoutSeconds) when is_binary(ContainerName), is_integer(TimeoutSeconds), TimeoutSeconds >= 0 ->
+    Url = build_stop_container_url(ContainerName, TimeoutSeconds),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_client:request("POST", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, 304, _Headers, _} ->
+            {error, <<"container already stopped">>};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch json:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec kill_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+kill_container(ContainerName) when is_binary(ContainerName) ->
+    kill_container(ContainerName, <<>>).
+
+-spec kill_container(ContainerName :: binary(), Signal :: binary()) -> ok | {error, Reason :: binary()}.
+kill_container(ContainerName, Signal) when is_binary(ContainerName), is_binary(Signal) ->
+    Url = build_kill_container_url(ContainerName, Signal),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_client:request("POST", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch json:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec remove_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+remove_container(ContainerName) when is_binary(ContainerName) ->
+    remove_container(ContainerName, false, false).
+
+-spec remove_container(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> ok | {error, Reason :: binary()}.
+remove_container(ContainerName, Force, RemoveVolumes)
+  when is_binary(ContainerName), is_boolean(Force), is_boolean(RemoveVolumes) ->
+    Url = build_remove_container_url(ContainerName, Force, RemoveVolumes),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_client:request("DELETE", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, 304, _Headers, _} ->
+            {error, <<"container already stopped">>};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch json:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec get_containers() -> {ok, Containers :: [map()]} | {error, Reason :: binary()}.
+get_containers() ->
+    Url = "/containers/json?all=true",
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_client:request("GET", Url, <<>>, Headers) of
+        {ok, 200, _Headers, ContainersBin} ->
+            Containers = json:decode(ContainersBin),
+            {ok, Containers};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch json:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec inspect_container(ContainerId :: binary()) -> {ok, Json :: map()} | {error, Error :: any()}.
+inspect_container(ContainerId) when is_binary(ContainerId) ->
+    Url = lists:flatten(io_lib:format("/containers/~s/json", [binary_to_list(ContainerId)])),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_client:request("GET", Url, <<>>, Headers) of
+        {ok, 200, _Headers, Resp} ->
+            decode_container_inspect_summary(Resp);
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch json:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec decode_container_inspect_summary(binary()) -> {ok, map()} | {error, binary()}.
+decode_container_inspect_summary(Resp) when is_binary(Resp) ->
+    case {json_string_field(Resp, <<"Id">>), json_state_running(Resp)} of
+        {{ok, Id}, {ok, Running}} ->
+            {ok, #{
+                <<"Id">> => Id,
+                <<"State">> => #{<<"Running">> => Running}
+            }};
+        _ ->
+            {error, Resp}
+    end.
+
+-spec json_string_field(binary(), binary()) -> {ok, binary()} | error.
+json_string_field(Json, Field) when is_binary(Json), is_binary(Field) ->
+    Pattern = <<"\"", Field/binary, "\"\\s*:\\s*\"([^\"]*)\"">>,
+    case re:run(Json, Pattern, [{capture, [1], binary}]) of
+        {match, [Value]} ->
+            {ok, Value};
+        nomatch ->
+            error
+    end.
+
+-spec json_state_running(binary()) -> {ok, boolean()} | error.
+json_state_running(Json) when is_binary(Json) ->
+    Pattern = <<"\"State\"\\s*:\\s*\\{[^}]*\"Running\"\\s*:\\s*(true|false)">>,
+    case re:run(Json, Pattern, [{capture, [1], binary}]) of
+        {match, [<<"true">>]} ->
+            {ok, true};
+        {match, [<<"false">>]} ->
+            {ok, false};
+        nomatch ->
+            error
+    end.
+
+-spec display_options(Options :: map()) -> ok.
+display_options(Options) when is_map(Options) ->
+    logger:debug("deploy options: ~p", [iolist_to_binary(json:encode(Options))]),
+    %lists:foreach(fun({K, V}) -> logger:debug("~p => ~p", [K, V]) end, maps:to_list(Options)),
+    ok.
+
+-spec build_stop_container_url(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> string().
+build_stop_container_url(ContainerName, 0) ->
+    lists:flatten(io_lib:format("/containers/~s/stop", [binary_to_list(ContainerName)]));
+build_stop_container_url(ContainerName, TimeoutSeconds) ->
+    lists:flatten(io_lib:format("/containers/~s/stop?t=~B", [binary_to_list(ContainerName), TimeoutSeconds])).
+
+-spec build_kill_container_url(ContainerName :: binary(), Signal :: binary()) -> string().
+build_kill_container_url(ContainerName, <<>>) ->
+    lists:flatten(io_lib:format("/containers/~s/kill", [binary_to_list(ContainerName)]));
+build_kill_container_url(ContainerName, Signal) ->
+    lists:flatten(io_lib:format("/containers/~s/kill?signal=~s", [binary_to_list(ContainerName), binary_to_list(Signal)])).
+
+-spec build_remove_container_url(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> string().
+build_remove_container_url(ContainerName, Force, RemoveVolumes) ->
+    ForceValue = boolean_to_query_value(Force),
+    RemoveVolumesValue = boolean_to_query_value(RemoveVolumes),
+    lists:flatten(io_lib:format("/containers/~s?force=~s&v=~s", [
+        binary_to_list(ContainerName),
+        ForceValue,
+        RemoveVolumesValue
+    ])).
+
+-spec boolean_to_query_value(boolean()) -> string().
+boolean_to_query_value(true) ->
+    "true";
+boolean_to_query_value(false) ->
+    "false".

apps/docker/src/docker_commands_tests.erl

@@ -0,0 +1,321 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 23. 9月 2025 17:23
+%%%-------------------------------------------------------------------
+-module(docker_commands_tests).
+-author("anlicheng").
+
+%% API
+-export([
+    test_all/0,
+    test_pull/0,
+    test_check_image_exist/0,
+    test_check_image_not_exist/0,
+    test_create_container/0,
+    test_create_container_without_create_options/0,
+    test_create_container_patches_options/0,
+    test_check_container_exist/0,
+    test_check_container_not_exist/0,
+    test_start_container/0,
+    test_stop_container/0,
+    test_stop_container_with_timeout/0,
+    test_kill_container/0,
+    test_kill_container_with_signal/0,
+    test_remove_container/0,
+    test_remove_container_with_options/0,
+    test_get_containers/0
+]).
+
+-define(TEST_IMAGE, <<"docker.1ms.run/library/nginx:latest">>).
+-define(TEST_CMD, [<<"nginx">>, <<"-g">>, <<"daemon off;">>]).
+
+-spec test_all() -> ok.
+test_all() ->
+    ok = test_pull(),
+    ok = test_check_image_exist(),
+    ok = test_check_image_not_exist(),
+    ok = test_create_container(),
+    ok = test_create_container_without_create_options(),
+    ok = test_create_container_patches_options(),
+    ok = test_check_container_exist(),
+    ok = test_check_container_not_exist(),
+    ok = test_start_container(),
+    ok = test_stop_container(),
+    ok = test_stop_container_with_timeout(),
+    ok = test_kill_container(),
+    ok = test_kill_container_with_signal(),
+    ok = test_remove_container(),
+    ok = test_remove_container_with_options(),
+    ok = test_get_containers(),
+    ok.
+
+-spec test_pull() -> ok.
+test_pull() ->
+    {ok, Ref, Pid, MRef} = docker_commands:pull_image(?TEST_IMAGE),
+    await_pull(Ref, Pid, MRef).
+
+-spec await_pull(reference(), pid(), reference()) -> ok.
+await_pull(Ref, Pid, MRef) ->
+    receive
+        {docker_client, Ref, {response, _Status, _Headers}} ->
+            await_pull(Ref, Pid, MRef);
+        {docker_client, Ref, {data, Data}} ->
+            logger:debug("msg is: ~p", [Data]),
+            await_pull(Ref, Pid, MRef);
+        {docker_client, Ref, done} ->
+            erlang:demonitor(MRef, [flush]),
+            ok;
+        {docker_client, Ref, {error, Reason}} ->
+            erlang:demonitor(MRef, [flush]),
+            error({pull_failed, Reason});
+        {'DOWN', MRef, process, Pid, Reason} ->
+            error({pull_client_down, Reason})
+    end.
+
+-spec test_check_image_exist() -> ok.
+test_check_image_exist() ->
+    true = docker_commands:check_image_exist(?TEST_IMAGE),
+    ok.
+
+-spec test_check_image_not_exist() -> ok.
+test_check_image_not_exist() ->
+    false = docker_commands:check_image_exist(<<"docker.1ms.run/library/not-exists-for-efka-tests:latest">>),
+    ok.
+
+-spec test_create_container() -> ok.
+test_create_container() ->
+    Name = test_container_name(<<"create">>),
+    ContainerDir = prepare_container_dir(Name),
+    {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)),
+    true = is_binary(ContainerId),
+    ok.
+
+-spec test_create_container_without_create_options() -> ok.
+test_create_container_without_create_options() ->
+    Name = test_container_name(<<"create-default">>),
+    ContainerDir = prepare_container_dir(Name),
+    Options = docker_container_builder:build_options(Name, ContainerDir, undefined),
+    assert_patched_default_options(Name, ContainerDir, Options),
+    ok.
+
+-spec test_create_container_patches_options() -> ok.
+test_create_container_patches_options() ->
+    Name = test_container_name(<<"create-patch">>),
+    ContainerDir = prepare_container_dir(Name),
+    Create = #{
+        config => #{
+            image => ?TEST_IMAGE,
+            cmd => ?TEST_CMD,
+            env => [<<"EXISTING_ENV=1">>],
+            volumes => [<<"/data">>]
+        },
+        host_config => #{
+            binds => [<<"/tmp:/tmp">>]
+        }
+    },
+    Params = #{
+        container_name => Name,
+        create => Create
+    },
+    Options = docker_container_builder:build_options(Name, ContainerDir, Create),
+    assert_patched_default_options(Name, ContainerDir, Options),
+    assert_existing_options_preserved(Options),
+    try
+        ok = test_pull(),
+        {ok, _ContainerId} = docker_commands:create_container(ContainerDir, Params),
+        ok
+    after
+        ok
+        %cleanup_container(Name)
+    end.
+
+-spec test_check_container_exist() -> ok.
+test_check_container_exist() ->
+    Name = test_container_name(<<"exist">>),
+    with_created_container(Name, fun(_ContainerDir, _ContainerId) ->
+        true = docker_commands:check_container_exist(Name),
+        ok
+    end).
+
+-spec test_check_container_not_exist() -> ok.
+test_check_container_not_exist() ->
+    false = docker_commands:check_container_exist(test_container_name(<<"missing">>)),
+    ok.
+
+-spec test_start_container() -> ok.
+test_start_container() ->
+    Name = test_container_name(<<"start">>),
+    with_created_container(Name, fun(_ContainerDir, ContainerId) ->
+        ok = docker_commands:start_container(Name),
+        true = docker_commands:is_container_running(ContainerId),
+        ok
+    end).
+
+-spec test_stop_container() -> ok.
+test_stop_container() ->
+    Name = test_container_name(<<"stop">>),
+    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
+        ok = docker_commands:stop_container(Name),
+        false = docker_commands:is_container_running(ContainerId),
+        ok
+    end).
+
+-spec test_stop_container_with_timeout() -> ok.
+test_stop_container_with_timeout() ->
+    Name = test_container_name(<<"stop-timeout">>),
+    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
+        ok = docker_commands:stop_container(Name, 1),
+        false = docker_commands:is_container_running(ContainerId),
+        ok
+    end).
+
+-spec test_kill_container() -> ok.
+test_kill_container() ->
+    Name = test_container_name(<<"kill">>),
+    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
+        ok = docker_commands:kill_container(Name),
+        timer:sleep(200),
+        false = docker_commands:is_container_running(ContainerId),
+        ok
+    end).
+
+-spec test_kill_container_with_signal() -> ok.
+test_kill_container_with_signal() ->
+    Name = test_container_name(<<"kill-signal">>),
+    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
+        ok = docker_commands:kill_container(Name, <<"SIGKILL">>),
+        timer:sleep(200),
+        false = docker_commands:is_container_running(ContainerId),
+        ok
+    end).
+
+-spec test_remove_container() -> ok.
+test_remove_container() ->
+    Name = test_container_name(<<"remove">>),
+    with_created_container(Name, fun(_ContainerDir, _ContainerId) ->
+        ok = docker_commands:remove_container(Name),
+        false = docker_commands:check_container_exist(Name),
+        ok
+    end, false).
+
+-spec test_remove_container_with_options() -> ok.
+test_remove_container_with_options() ->
+    Name = test_container_name(<<"remove-opts">>),
+    with_started_container(Name, fun(_ContainerDir, _ContainerId) ->
+        ok = docker_commands:remove_container(Name, true, false),
+        false = docker_commands:check_container_exist(Name),
+        ok
+    end, false).
+
+-spec test_get_containers() -> ok.
+test_get_containers() ->
+    Name = test_container_name(<<"list">>),
+    with_created_container(Name, fun(_ContainerDir, ContainerId) ->
+        {ok, Containers} = docker_commands:get_containers(),
+        logger:debug("list containers: ~p", [Containers]),
+        true = is_list(Containers),
+        true = contains_container(Name, ContainerId, Containers),
+        ok
+    end).
+
+-spec with_created_container(binary(), fun((string(), binary()) -> ok)) -> ok.
+with_created_container(Name, Fun) ->
+    with_created_container(Name, Fun, true).
+
+-spec with_created_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok.
+with_created_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) ->
+    ContainerDir = prepare_container_dir(Name),
+    try
+        ok = test_pull(),
+        {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)),
+        ok = Fun(ContainerDir, ContainerId)
+    after
+        case Cleanup of
+            true ->
+                cleanup_container(Name);
+            false ->
+                ok
+        end
+    end.
+
+-spec with_started_container(binary(), fun((string(), binary()) -> ok)) -> ok.
+with_started_container(Name, Fun) ->
+    with_started_container(Name, Fun, true).
+
+-spec with_started_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok.
+with_started_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) ->
+    with_created_container(Name, fun(ContainerDir, ContainerId) ->
+        ok = docker_commands:start_container(Name),
+        ok = Fun(ContainerDir, ContainerId)
+    end, Cleanup).
+
+-spec minimal_params(binary()) -> map().
+minimal_params(Name) when is_binary(Name) ->
+    #{
+        container_name => Name,
+        create => #{
+            config => #{
+                image => ?TEST_IMAGE,
+                cmd => ?TEST_CMD
+            }
+        }
+    }.
+
+-spec prepare_container_dir(binary()) -> string().
+prepare_container_dir(Name) when is_binary(Name) ->
+    Dir = lists:flatten(io_lib:format("/usr/local/code/efka/~ts/", [Name])),
+    ok = filelib:ensure_dir(Dir ++ "placeholder"),
+    ok = file:write_file(Dir ++ "service.conf", <<>>, [write]),
+    Dir.
+
+-spec cleanup_container(binary()) -> ok.
+cleanup_container(Name) when is_binary(Name) ->
+    _ = docker_commands:remove_container(Name, true, false),
+    ok.
+
+-spec assert_patched_default_options(binary(), string(), map()) -> ok.
+assert_patched_default_options(Name, ContainerDir, Options)
+  when is_binary(Name), is_list(ContainerDir), is_map(Options) ->
+    ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)),
+    ExpectedBind = <<ConfigFile/binary, ":/usr/local/etc/service.conf">>,
+    #{<<"Env">> := Env,
+        <<"Volumes">> := Volumes,
+        <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options,
+    true = lists:member(<<"CONTAINER_NAME=", Name/binary>>, Env),
+    true = maps:is_key(<<"/usr/local/etc/service.conf">>, Volumes),
+    true = lists:member(ExpectedBind, Binds),
+    ok.
+
+-spec assert_existing_options_preserved(map()) -> ok.
+assert_existing_options_preserved(Options) when is_map(Options) ->
+    #{<<"Env">> := Env,
+        <<"Volumes">> := Volumes,
+        <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options,
+    true = lists:member(<<"EXISTING_ENV=1">>, Env),
+    true = maps:is_key(<<"/data">>, Volumes),
+    true = lists:member(<<"/tmp:/tmp">>, Binds),
+    ok.
+
+-spec contains_container(binary(), binary(), [map()]) -> boolean().
+contains_container(Name, ContainerId, Containers) when is_binary(Name), is_binary(ContainerId), is_list(Containers) ->
+    lists:any(fun(Container) -> container_matches(Name, ContainerId, Container) end, Containers).
+
+-spec container_matches(binary(), binary(), map()) -> boolean().
+container_matches(Name, ContainerId, #{<<"Id">> := Id, <<"Names">> := Names}) when is_binary(Id), is_list(Names) ->
+    lists:member(<<"/", Name/binary>>, Names) orelse has_id_prefix(ContainerId, Id);
+container_matches(_Name, _ContainerId, _Container) ->
+    false.
+
+-spec has_id_prefix(binary(), binary()) -> boolean().
+has_id_prefix(ExpectedId, ActualId) when is_binary(ExpectedId), is_binary(ActualId) ->
+    PrefixLen = erlang:min(byte_size(ExpectedId), byte_size(ActualId)),
+    binary:part(ExpectedId, 0, PrefixLen) =:= binary:part(ActualId, 0, PrefixLen).
+
+-spec test_container_name(binary()) -> binary().
+test_container_name(Prefix) when is_binary(Prefix) ->
+    Suffix = integer_to_binary(erlang:unique_integer([positive])),
+    <<"efka-test-", Prefix/binary, "-", Suffix/binary>>.

apps/docker/src/docker_container_builder.erl

@@ -0,0 +1,379 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 21. 4月 2026 15:58
+%%%-------------------------------------------------------------------
+-module(docker_container_builder).
+-author("anlicheng").
+
+%% API
+-export([build_options/3]).
+
+-spec build_options(binary(), string(), map() | undefined) -> map().
+build_options(ContainerName, ContainerDir, Create0) when is_binary(ContainerName), is_list(ContainerDir) ->
+    ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)),
+    Create = patch_create_options(ContainerName, ConfigFile, Create0),
+    build_create_options(Create).
+
+-spec patch_create_options(binary(), binary(), map() | undefined) -> map().
+patch_create_options(ContainerName, ConfigFile, undefined) ->
+    patch_create_options(ContainerName, ConfigFile, #{});
+patch_create_options(ContainerName, ConfigFile, Create0)
+  when is_binary(ContainerName), is_binary(ConfigFile), is_map(Create0) ->
+    Config = patch_container_config(ContainerName, ensure_container_config(field(Create0, <<"config">>, undefined))),
+    HostConfig = patch_host_config(ConfigFile, ensure_host_config(field(Create0, <<"host_config">>, undefined))),
+    Create0#{
+        <<"config">> => Config,
+        <<"host_config">> => HostConfig
+    }.
+
+-spec patch_container_config(binary(), map()) -> map().
+patch_container_config(ContainerName, Config0) when is_binary(ContainerName), is_map(Config0) ->
+    Env0 = [to_binary(EnvItem) || EnvItem <- field(Config0, <<"env">>, [])],
+    Volumes0 = [to_binary(Volume) || Volume <- field(Config0, <<"volumes">>, [])],
+    ConfigVolume = <<"/usr/local/etc/service.conf">>,
+    Envs = add_unique_front([<<"CONTAINER_NAME=", ContainerName/binary>>], Env0),
+    Volumes = add_unique_front([ConfigVolume], Volumes0),
+    Config0#{
+        <<"env">> => Envs,
+        <<"volumes">> => Volumes
+    }.
+
+-spec patch_host_config(binary(), map()) -> map().
+patch_host_config(ConfigFile, HostConfig0) when is_binary(ConfigFile), is_map(HostConfig0) ->
+    Binds0 = [to_binary(Bind) || Bind <- field(HostConfig0, <<"binds">>, [])],
+    ConfigBind = <<ConfigFile/binary, ":/usr/local/etc/service.conf">>,
+    HostConfig0#{<<"binds">> => add_unique_front([ConfigBind], Binds0)}.
+
+-spec ensure_container_config(map() | undefined) -> map().
+ensure_container_config(undefined) ->
+    #{};
+ensure_container_config(Config) when is_map(Config) ->
+    Config.
+
+-spec ensure_host_config(map() | undefined) -> map().
+ensure_host_config(undefined) ->
+    #{};
+ensure_host_config(HostConfig) when is_map(HostConfig) ->
+    HostConfig.
+
+-spec build_create_options(map()) -> map().
+build_create_options(Create0) when is_map(Create0) ->
+    Config = field(Create0, <<"config">>, #{}),
+    HostConfig = field(Create0, <<"host_config">>, #{}),
+    Endpoints = networking_config_endpoints(field(Create0, <<"networking_config">>, undefined)),
+    #{
+        <<"Image">> => to_binary(field(Config, <<"image">>, <<>>)),
+        <<"Cmd">> => [to_binary(CommandItem) || CommandItem <- field(Config, <<"cmd">>, [])],
+        <<"Entrypoint">> => [to_binary(EntrypointItem) || EntrypointItem <- field(Config, <<"entrypoint">>, [])],
+        <<"Env">> => [to_binary(EnvItem) || EnvItem <- field(Config, <<"env">>, [])],
+        <<"Labels">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(field(Config, <<"labels">>, #{}))]),
+        <<"Volumes">> => build_volumes(field(Config, <<"volumes">>, [])),
+        <<"User">> => to_binary(field(Config, <<"user">>, <<>>)),
+        <<"WorkingDir">> => to_binary(field(Config, <<"working_dir">>, <<>>)),
+        <<"Hostname">> => to_binary(field(Config, <<"hostname">>, <<>>)),
+        <<"ExposedPorts">> => build_expose(field(Config, <<"exposed_ports">>, [])),
+        <<"NetworkingConfig">> => build_networking_config(Endpoints),
+        <<"Healthcheck">> => build_healthcheck(field(Config, <<"healthcheck">>, undefined)),
+        <<"HostConfig">> => fold_merge([
+            build_binds(field(HostConfig, <<"binds">>, [])),
+            build_network_mode(field(HostConfig, <<"network_mode">>, <<>>)),
+            build_restart(field(HostConfig, <<"restart_policy">>, undefined)),
+            build_privileged(field(HostConfig, <<"privileged">>, false)),
+            build_cap_add_drop(field(HostConfig, <<"cap_add">>, []), field(HostConfig, <<"cap_drop">>, [])),
+            build_devices(field(HostConfig, <<"devices">>, [])),
+            build_resources(
+                field(HostConfig, <<"memory">>, 0),
+                field(HostConfig, <<"memory_reservation">>, 0),
+                field(HostConfig, <<"nano_cpus">>, 0),
+                field(HostConfig, <<"cpu_shares">>, 0)
+            ),
+            build_port_bindings(field(HostConfig, <<"port_bindings">>, [])),
+            build_ulimits(field(HostConfig, <<"ulimits">>, [])),
+            build_tmpfs(field(HostConfig, <<"tmpfs">>, #{})),
+            build_sysctls(field(HostConfig, <<"sysctls">>, #{})),
+            build_extra_hosts(field(HostConfig, <<"extra_hosts">>, []))
+        ])
+    }.
+
+-spec networking_config_endpoints(map() | undefined) -> [map()].
+networking_config_endpoints(undefined) ->
+    [];
+networking_config_endpoints(NetworkingConfig) when is_map(NetworkingConfig) ->
+    field(NetworkingConfig, <<"endpoints">>, []).
+
+-spec fold_merge([map()]) -> map().
+fold_merge(List) ->
+    lists:foldl(fun maps:merge/2, #{}, List).
+
+-spec build_expose([map()]) -> map().
+build_expose(Ports) when is_list(Ports) ->
+    case Ports of
+        [] ->
+            #{};
+        _ ->
+            maps:from_list([{normalize_expose_port(Port), #{}} || Port <- Ports])
+    end.
+
+-spec build_volumes([binary() | list() | atom()]) -> map().
+build_volumes(Volumes) when is_list(Volumes) ->
+    case Volumes of
+        [] ->
+            #{};
+        _ ->
+            maps:from_list([{to_binary(ContainerPath), #{}} || ContainerPath <- Volumes])
+    end.
+
+-spec build_binds([binary() | list() | atom()]) -> map().
+build_binds(Binds) when is_list(Binds) ->
+    case Binds of
+        [] ->
+            #{};
+        _ ->
+            #{<<"Binds">> => [to_binary(Bind) || Bind <- Binds]}
+    end.
+
+-spec build_networking_config([map()]) -> map().
+build_networking_config(Endpoints) when is_list(Endpoints) ->
+    case Endpoints of
+        [] ->
+            #{};
+        _ ->
+            NetCfg = maps:from_list([{to_binary(Name), #{}} || #{<<"name">> := Name} <- Endpoints]),
+            #{<<"EndpointsConfig">> => NetCfg}
+    end.
+
+-spec build_network_mode(binary() | list() | atom()) -> map().
+build_network_mode(<<>>) ->
+    #{};
+build_network_mode(NetworkMode) ->
+    #{<<"NetworkMode">> => to_binary(NetworkMode)}.
+
+-spec build_healthcheck(map() | undefined) -> map().
+build_healthcheck(undefined) ->
+    #{};
+build_healthcheck(Healthcheck) when is_map(Healthcheck) ->
+    #{
+        <<"Test">> => [to_binary(Item) || Item <- field(Healthcheck, <<"test">>, [])],
+        <<"Interval">> => field(Healthcheck, <<"interval_ns">>, 0),
+        <<"Timeout">> => field(Healthcheck, <<"timeout_ns">>, 0),
+        <<"Retries">> => field(Healthcheck, <<"retries">>, 0)
+    }.
+
+-spec build_restart(map() | undefined) -> map().
+build_restart(undefined) ->
+    #{};
+build_restart(RestartPolicy0) when is_map(RestartPolicy0) ->
+    RestartPolicy = #{
+        <<"Name">> => to_binary(field(RestartPolicy0, <<"name">>, <<>>))
+    },
+    case field(RestartPolicy0, <<"maximum_retry_count">>, 0) of
+        0 ->
+            #{<<"RestartPolicy">> => RestartPolicy};
+        RetryCount ->
+            #{<<"RestartPolicy">> => RestartPolicy#{
+                <<"MaximumRetryCount">> => RetryCount
+            }}
+    end.
+
+-spec build_privileged(any()) -> map().
+build_privileged(Privileged) ->
+    case to_bool(Privileged) of
+        true ->
+            #{<<"Privileged">> => true};
+        _ ->
+            #{}
+    end.
+
+-spec build_cap_add_drop([binary() | list() | atom()], [binary() | list() | atom()]) -> map().
+build_cap_add_drop(Add, Drop) when is_list(Add), is_list(Drop) ->
+    case {Add, Drop} of
+        {[], []} ->
+            #{};
+        _ ->
+            #{
+                <<"CapAdd">> => [to_binary(Item) || Item <- Add],
+                <<"CapDrop">> => [to_binary(Item) || Item <- Drop]
+            }
+    end.
+
+-spec build_devices([map()]) -> map().
+build_devices(Devices) when is_list(Devices) ->
+    case Devices of
+        [] ->
+            #{};
+        _ ->
+            DevObjs = [#{
+                <<"PathOnHost">> => to_binary(HostPath),
+                <<"PathInContainer">> => to_binary(ContainerPath),
+                <<"CgroupPermissions">> => device_permissions(Permissions)
+            } || #{
+                <<"path_on_host">> := HostPath,
+                <<"path_in_container">> := ContainerPath,
+                <<"cgroup_permissions">> := Permissions
+            } <- Devices],
+            #{<<"Devices">> => DevObjs}
+    end.
+
+-spec build_resources(integer(), integer(), integer(), integer()) -> map().
+build_resources(MemoryBytes, ReservationBytes, NanoCpus, CpuShares) ->
+    HostConfig0 = #{},
+    HostConfig1 = case MemoryBytes of
+        0 ->
+            HostConfig0;
+        _ ->
+            HostConfig0#{<<"Memory">> => MemoryBytes}
+    end,
+    HostConfig2 = case ReservationBytes of
+        0 ->
+            HostConfig1;
+        _ ->
+            HostConfig1#{<<"MemoryReservation">> => ReservationBytes}
+    end,
+    HostConfig3 = case NanoCpus of
+        0 ->
+            HostConfig2;
+        _ ->
+            HostConfig2#{<<"NanoCpus">> => NanoCpus}
+    end,
+    case CpuShares of
+        0 ->
+            HostConfig3;
+        _ ->
+            HostConfig3#{<<"CpuShares">> => CpuShares}
+    end.
+
+-spec build_port_bindings([map()]) -> map().
+build_port_bindings(PortBindings) when is_list(PortBindings) ->
+    case PortBindings of
+        [] ->
+            #{};
+        _ ->
+            #{<<"PortBindings">> => fold_port_bindings(PortBindings)}
+    end.
+
+-spec fold_port_bindings([map()]) -> map().
+fold_port_bindings(PortBindings) ->
+    lists:foldl(
+        fun(PortBinding, Acc) ->
+            PortKey = port_binding_key(PortBinding),
+            Binding = port_binding_value(PortBinding),
+            Existing = maps:get(PortKey, Acc, []),
+            Acc#{PortKey => Existing ++ [Binding]}
+        end,
+        #{},
+        PortBindings).
+
+-spec port_binding_key(map()) -> binary().
+port_binding_key(#{<<"container_port">> := ContainerPort, <<"protocol">> := Protocol}) ->
+    PortBin = integer_to_binary(ContainerPort),
+    ProtocolBin = to_binary(Protocol),
+    case ProtocolBin of
+        <<>> ->
+            <<PortBin/binary, "/tcp">>;
+        <<"tcp">> ->
+            <<PortBin/binary, "/tcp">>;
+        _ ->
+            <<PortBin/binary, "/", ProtocolBin/binary>>
+    end.
+
+-spec port_binding_value(map()) -> map().
+port_binding_value(#{<<"host_port">> := HostPort} = PortBinding) ->
+    HostIp = to_binary(field(PortBinding, <<"host_ip">>, <<>>)),
+    #{<<"HostIp">> => HostIp, <<"HostPort">> => integer_to_binary(HostPort)}.
+
+-spec build_ulimits([map()]) -> map().
+build_ulimits(Ulimits) when is_list(Ulimits) ->
+    case Ulimits of
+        [] ->
+            #{};
+        _ ->
+            #{<<"Ulimits">> => [#{
+                <<"Name">> => to_binary(Name),
+                <<"Soft">> => Soft,
+                <<"Hard">> => Hard
+            } || #{<<"name">> := Name, <<"soft">> := Soft, <<"hard">> := Hard} <- Ulimits]}
+    end.
+
+-spec build_sysctls(map()) -> map().
+build_sysctls(Sysctls) when is_map(Sysctls) ->
+    case maps:size(Sysctls) of
+        0 ->
+            #{};
+        _ ->
+            #{<<"Sysctls">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(Sysctls)])}
+    end.
+
+-spec build_tmpfs(map()) -> map().
+build_tmpfs(Tmpfs) when is_map(Tmpfs) ->
+    case maps:size(Tmpfs) of
+        0 ->
+            #{};
+        _ ->
+            #{<<"Tmpfs">> => maps:from_list([{to_binary(Path), to_binary(Options)} || {Path, Options} <- maps:to_list(Tmpfs)])}
+    end.
+
+-spec build_extra_hosts([binary() | list() | atom()]) -> map().
+build_extra_hosts(Hosts) when is_list(Hosts) ->
+    case Hosts of
+        [] ->
+            #{};
+        _ ->
+            #{<<"ExtraHosts">> => [to_binary(Host) || Host <- Hosts]}
+    end.
+
+-spec normalize_expose_port(map()) -> binary().
+normalize_expose_port(#{<<"container_port">> := Port, <<"protocol">> := Protocol}) ->
+    PortBin = integer_to_binary(Port),
+    ProtocolBin = to_binary(Protocol),
+    case ProtocolBin of
+        <<>> ->
+            <<PortBin/binary, "/tcp">>;
+        <<"tcp">> ->
+            <<PortBin/binary, "/tcp">>;
+        _ ->
+            <<PortBin/binary, "/", ProtocolBin/binary>>
+    end.
+
+-spec device_permissions(binary() | list() | atom()) -> binary().
+device_permissions(<<>>) ->
+    <<"rwm">>;
+device_permissions(Permissions) ->
+    to_binary(Permissions).
+
+-spec add_unique_front([binary()], [binary()]) -> [binary()].
+add_unique_front([], List) ->
+    List;
+add_unique_front([Item | Rest], List) ->
+    NList = case lists:member(Item, List) of
+        true ->
+            List;
+        false ->
+            [Item | List]
+    end,
+    add_unique_front(Rest, NList).
+
+-spec field(map(), binary(), term()) -> term().
+field(Map, Key, Default) when is_map(Map), is_binary(Key) ->
+    maps:get(Key, Map, Default).
+
+-spec to_binary(binary() | list() | atom() | any()) -> binary().
+to_binary(Value) when is_binary(Value) ->
+    Value;
+to_binary(Value) when is_list(Value) ->
+    list_to_binary(Value);
+to_binary(Value) when is_atom(Value) ->
+    atom_to_binary(Value, utf8);
+to_binary(Value) ->
+    iolist_to_binary(io_lib:format("~p", [Value])).
+
+-spec to_bool(true | false | 0 | 1 | undefined) -> boolean().
+to_bool(true) ->
+    true;
+to_bool(1) ->
+    true;
+to_bool(_) ->
+    false.

apps/docker/src/docker_deploy_manager.erl

@@ -0,0 +1,90 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2026, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 20. 4月 2026
+%%%-------------------------------------------------------------------
+-module(docker_deploy_manager).
+-author("anlicheng").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0, deploy/2]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+-record(state, {
+    root_dir :: string(),
+    task_map = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec start_link() -> {ok, pid()} | ignore | {error, term()}.
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+-spec deploy(integer(), map()) -> ok | {error, binary()}.
+deploy(TaskId, Params) when is_integer(TaskId), is_map(Params) ->
+    gen_server:call(?SERVER, {deploy, TaskId, Params}).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+-spec init(list()) -> {ok, #state{}}.
+init([]) ->
+    erlang:process_flag(trap_exit, true),
+    {ok, RootDir} = docker_helper:root_dir(),
+    {ok, #state{root_dir = RootDir}}.
+
+-spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, term(), #state{}}.
+handle_call({deploy, TaskId, Params}, _From, State = #state{root_dir = RootDir, task_map = TaskMap})
+  when is_map(Params) ->
+    ContainerName = maps:get(<<"container_name">>, Params),
+    {ok, ContainerDir} = docker_helper:ensure_container_dir(RootDir, ContainerName),
+    {ok, {TaskPid, _Ref}} = docker_deployer:start_monitor(TaskId, ContainerDir, Params),
+    logger:debug("[docker_deploy_manager] start deploy task_id: ~p, params: ~p", [TaskId, Params]),
+    {reply, ok, State#state{task_map = maps:put(TaskPid, TaskId, TaskMap)}};
+handle_call(_Request, _From, State) ->
+    {reply, ok, State}.
+
+-spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
+handle_cast(_Request, State) ->
+    {noreply, State}.
+
+-spec handle_info(term(), #state{}) -> {noreply, #state{}}.
+handle_info({'DOWN', _Ref, process, TaskPid, Reason}, State = #state{task_map = TaskMap}) ->
+    case maps:take(TaskPid, TaskMap) of
+        error ->
+            {noreply, State};
+        {TaskId, NTaskMap} ->
+            case Reason of
+                normal ->
+                    logger:debug("[docker_deploy_manager] task_id: ~p, exit normal", [TaskId]);
+                Error0 ->
+                    Error = iolist_to_binary(io_lib:format("~p", [Error0])),
+                    docker_task_reporter:stream(TaskId, <<"error">>, <<"任务失败: "/utf8, Error/binary>>),
+                    docker_task_reporter:close(TaskId, <<"task exited">>),
+                    logger:notice("[docker_deploy_manager] task_id: ~p, exit with error: ~p", [TaskId, Error])
+            end,
+            {noreply, State#state{task_map = NTaskMap}}
+    end;
+handle_info(_Info, State) ->
+    {noreply, State}.
+
+-spec terminate(term(), #state{}) -> ok.
+terminate(_Reason, _State) ->
+    ok.
+
+-spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
+code_change(_OldVsn, State, _Extra) ->
+    {ok, State}.

apps/docker/src/docker_deployer.erl

@@ -0,0 +1,211 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 07. 5月 2025 15:47
+%%%-------------------------------------------------------------------
+-module(docker_deployer).
+-author("anlicheng").
+-dialyzer([{nowarn_function, normalize_image/1}]).
+
+%% API
+-export([start_monitor/3]).
+-export([deploy/3]).
+
+-define(TASK_SUCCESS, <<"success">>).
+-define(TASK_FAIL, <<"fail">>).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec(start_monitor(TaskId :: integer(), ContainerDir :: string(), Params :: map()) ->
+    {ok, {pid(), reference()}}).
+start_monitor(TaskId, ContainerDir, Params)
+  when is_integer(TaskId), is_list(ContainerDir), is_map(Params) ->
+    {ok, spawn_monitor(?MODULE, deploy, [TaskId, ContainerDir, Params])}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+%{
+%   "image": "nginx:latest",
+%   "container_name": "my_nginx",
+%   "ports": ["8080:80", "443:443"],
+%   "volumes": ["/host/data:/data", "/host/log:/var/log"],
+%   "envs": ["ENV1=val1", "ENV2=val2"],
+%   "entrypoint": ["/docker-entrypoint.sh"],
+%   "command": ["nginx", "-g", "daemon off;"],
+%   "restart": "always"
+%}
+-spec deploy(TaskId :: integer(), ContainerDir :: string(), Params :: map()) -> ok.
+deploy(TaskId, ContainerDir, Params) when is_integer(TaskId), is_list(ContainerDir), is_map(Params) ->
+    ContainerName = deploy_container_name(Params),
+    Image0 = deploy_image(Params),
+    report_task_event(TaskId, <<"info">>, <<"开始部署容器:"/utf8, ContainerName/binary>>),
+    try
+        ok = ensure_container_absent(TaskId, ContainerName),
+        {ok, Image} = ensure_image_ready(TaskId, Image0),
+        {ok, ContainerId} = create_container_and_config(TaskId, ContainerDir, Params),
+        ShortContainerId = short_container_id(ContainerId),
+        report_task_event(TaskId, <<"info">>, <<"容器创建成功: "/utf8, ShortContainerId/binary>>),
+        report_task_event(TaskId, <<"info">>, <<"任务完成"/utf8>>),
+        write_task_summary(TaskId, <<"success">>, ContainerName, Image, ContainerId),
+        docker_task_reporter:close(TaskId, ?TASK_SUCCESS)
+    catch
+        throw:{deploy_error, Reason} ->
+            report_task_event(TaskId, <<"error">>, Reason),
+            report_task_event(TaskId, <<"error">>, <<"任务失败"/utf8>>),
+            write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined),
+            write_task_failure_reason(TaskId, Reason),
+            docker_task_reporter:close(TaskId, ?TASK_FAIL);
+        Class:Reason:Stacktrace ->
+            Error = iolist_to_binary(io_lib:format("deploy crashed: ~p:~p ~p", [Class, Reason, Stacktrace])),
+            report_task_event(TaskId, <<"error">>, Error),
+            report_task_event(TaskId, <<"error">>, <<"任务失败"/utf8>>),
+            write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined),
+            write_task_failure_reason(TaskId, Error),
+            docker_task_reporter:close(TaskId, ?TASK_FAIL)
+    end.
+
+-spec normalize_image(binary()) -> binary().
+normalize_image(Image) when is_binary(Image) ->
+    Parts = binary:split(Image, <<"/">>, [global]),
+    {PrefixParts, [Last]} = lists:split(length(Parts) - 1, Parts),
+    NormalizedLast = case binary:split(Last, <<":">>, [global]) of
+                         [_Name] -> <<Last/binary, ":latest">>;
+                         [_Name, _Tag] -> Last
+                     end,
+    iolist_to_binary(lists:join(<<"/">>, PrefixParts ++ [NormalizedLast])).
+
+-spec report_task_event(TaskId :: integer(), Level :: binary(), Msg :: binary()) -> ok.
+report_task_event(TaskId, Level, Msg) when is_integer(TaskId), is_binary(Level), is_binary(Msg) ->
+    docker_task_reporter:stream(TaskId, Level, Msg).
+
+-spec write_task_summary(integer(), binary(), binary(), binary(), undefined | binary()) -> ok.
+write_task_summary(TaskId, Status, ContainerName, Image, ContainerId)
+  when is_integer(TaskId), is_binary(Status), is_binary(ContainerName), is_binary(Image) ->
+    Fields0 = [
+        <<"type=deploy_summary">>,
+        <<"task_id=">>, integer_to_binary(TaskId),
+        <<" status=">>, Status,
+        <<" container_name=">>, ContainerName,
+        <<" image=">>, Image
+    ],
+    Fields = case ContainerId of
+                 undefined ->
+                     Fields0;
+                 ContainerId0 when is_binary(ContainerId0) ->
+                     Fields0 ++ [<<" container_id=">>, short_container_id(ContainerId0)]
+             end,
+    efka_logger:write(iolist_to_binary(Fields)).
+
+-spec write_task_failure_reason(integer(), binary()) -> ok.
+write_task_failure_reason(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) ->
+    Info = iolist_to_binary([
+        <<"type=deploy_failure_reason task_id=">>,
+        integer_to_binary(TaskId),
+        <<" reason=">>,
+        Reason
+    ]),
+    efka_logger:write(Info).
+
+-spec ensure_container_absent(TaskId :: integer(), ContainerName :: binary()) -> ok.
+ensure_container_absent(TaskId, ContainerName) when is_integer(TaskId), is_binary(ContainerName) ->
+    report_task_event(TaskId, <<"info">>, <<"开始创建容器: "/utf8, ContainerName/binary>>),
+    ok.
+
+-spec ensure_image_ready(TaskId :: integer(), Image0 :: binary()) -> {ok, binary()}.
+ensure_image_ready(TaskId, Image0) when is_integer(TaskId), is_binary(Image0) ->
+    Image = normalize_image(Image0),
+    report_task_event(TaskId, <<"info">>, <<"使用镜像:"/utf8, Image/binary>>),
+    report_task_event(TaskId, <<"info">>, <<"开始拉取镜像:"/utf8, Image/binary>>),
+    case docker_commands:pull_image(Image) of
+        {ok, Ref, Pid, MRef} ->
+            await_pull_image(TaskId, Ref, Pid, MRef),
+            {ok, Image}
+    end.
+
+-spec await_pull_image(TaskId :: integer(), Ref :: reference(), Pid :: pid(), MRef :: reference()) -> ok.
+await_pull_image(TaskId, Ref, Pid, MRef) ->
+    receive
+        {docker_client, Ref, {response, _Status, _Headers}} ->
+            await_pull_image(TaskId, Ref, Pid, MRef);
+        {docker_client, Ref, {data, Data}} ->
+            report_task_event(TaskId, <<"info">>, Data),
+            await_pull_image(TaskId, Ref, Pid, MRef);
+        {docker_client, Ref, done} ->
+            erlang:demonitor(MRef, [flush]),
+            ok;
+        {docker_client, Ref, {error, Reason}} ->
+            erlang:demonitor(MRef, [flush]),
+            report_task_event(TaskId, <<"error">>, Reason),
+            throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>});
+        {'DOWN', MRef, process, Pid, Reason0} ->
+            Reason = iolist_to_binary(io_lib:format("~p", [Reason0])),
+            throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>})
+    end.
+
+-spec create_container_and_config(TaskId :: integer(), ContainerDir :: string(), Params :: map()) ->
+    {ok, binary()}.
+create_container_and_config(TaskId, ContainerDir, Params)
+  when is_integer(TaskId), is_list(ContainerDir), is_map(Params) ->
+    case docker_commands:create_container(ContainerDir, Params) of
+        {ok, ContainerId} ->
+            ok = create_config_file(TaskId, ContainerDir),
+            {ok, ContainerId};
+        {error, Reason} when is_binary(Reason) ->
+            throw({deploy_error, format_create_container_error(Reason)});
+        {error, Reason} ->
+            Error = iolist_to_binary(io_lib:format("容器创建失败: ~p", [Reason])),
+            throw({deploy_error, Error})
+    end.
+
+-spec create_config_file(TaskId :: integer(), ContainerDir :: string()) -> ok.
+create_config_file(TaskId, ContainerDir) when is_integer(TaskId), is_list(ContainerDir) ->
+    ConfigFile = docker_helper:get_config_file(ContainerDir),
+    case file:open(ConfigFile, [write, exclusive]) of
+        {ok, FD} ->
+            ok = file:write(FD, <<>>),
+            file:close(FD),
+            ok;
+        {error, Reason} ->
+            ReasonBin = list_to_binary(io_lib:format("~p", [Reason])),
+            report_task_event(TaskId, <<"notice">>, <<"创建配置文件失败: "/utf8, ReasonBin/binary>>),
+            ok
+    end.
+
+-spec format_create_container_error(binary()) -> binary().
+format_create_container_error(Reason) when is_binary(Reason) ->
+    case is_container_already_exists_error(Reason) of
+        true ->
+            <<"本地容器已经存在"/utf8>>;
+        false ->
+            <<"容器创建失败: "/utf8, Reason/binary>>
+    end.
+
+-spec is_container_already_exists_error(binary()) -> boolean().
+is_container_already_exists_error(Reason) when is_binary(Reason) ->
+    binary:match(Reason, <<"is already in use by container">>) =/= nomatch orelse
+        binary:match(Reason, <<"Conflict. The container name ">>) =/= nomatch.
+
+-spec short_container_id(binary()) -> binary().
+short_container_id(ContainerId) when is_binary(ContainerId), byte_size(ContainerId) >= 12 ->
+    binary:part(ContainerId, 0, 12);
+short_container_id(ContainerId) when is_binary(ContainerId) ->
+    ContainerId.
+
+-spec deploy_container_name(map()) -> binary().
+deploy_container_name(#{<<"container_name">> := ContainerName}) when is_binary(ContainerName) ->
+    ContainerName;
+deploy_container_name(_) ->
+    throw({deploy_error, <<"invalid deploy params: container_name missing">>}).
+
+-spec deploy_image(map()) -> binary().
+deploy_image(#{<<"create">> := #{<<"config">> := #{<<"image">> := Image}}}) when is_binary(Image) ->
+    Image;
+deploy_image(_) ->
+    throw({deploy_error, <<"invalid deploy params: image missing">>}).

apps/docker/src/docker_events.erl

@@ -4,46 +4,35 @@
 %%% @doc
 %%%
 %%% @end
-%%% Created : 13. 8月 2025 16:05
+%%% Created : 16. 9月 2025 16:48
 %%%-------------------------------------------------------------------
--module(cache_model).
+-module(docker_events).
 -author("anlicheng").
--include("efka_tables.hrl").
 
 -behaviour(gen_server).
 
 %% API
 -export([start_link/0]).
--export([insert/2, fetch_next/0, delete/1, get_all_cache/0]).
+-export([monitor_container/2]).
 
 %% gen_server callbacks
 -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
 
 -define(SERVER, ?MODULE).
--define(TAB, cache).
 
 -record(state, {
-
+    port,
+    %% 观察者
+    monitors = #{}
 }).
 
 %%%===================================================================
 %%% API
 %%%===================================================================
 
--spec insert(Method :: integer(), Data :: binary()) -> ok | {error, Reason :: any()}.
-insert(Method, Data) when is_integer(Method), is_binary(Data) ->
-    Cache = #cache{id = generate_id(), method = Method, data = Data},
-    gen_server:call(?SERVER, {insert, Cache}).
-
-fetch_next() ->
-    gen_server:call(?SERVER, fetch_next).
-
-delete(Id) when is_integer(Id) ->
-    gen_server:call(?SERVER, {delete, Id}).
-
--spec get_all_cache() -> [#cache{}].
-get_all_cache() ->
-    gen_server:call(?SERVER, get_all_cache).
+-spec monitor_container(ReceiverPid :: pid(), ContainerId :: binary()) -> ok.
+monitor_container(ReceiverPid, ContainerId) when is_pid(ReceiverPid), is_binary(ContainerId) ->
+    gen_server:cast(?SERVER, {monitor_container, ReceiverPid, ContainerId}).
 
 %% @doc Spawns the server and registers the local name (unique)
 -spec(start_link() ->
@@ -61,9 +50,8 @@ start_link() ->
     {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term()} | ignore).
 init([]) ->
-    {ok, DetsDir} = application:get_env(efka, dets_dir),
-    File = DetsDir ++ "cache.dets",
-    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, bag}, {keypos, 2}]),
+    process_flag(trap_exit, true),
+    try_attach_events(0),
     {ok, #state{}}.
 
 %% @private
@@ -76,25 +64,6 @@ init([]) ->
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_call({insert, Cache}, _From, State = #state{}) ->
-    ok = dets:insert(?TAB, Cache),
-    {reply, ok, State};
-handle_call(fetch_next, _From, State = #state{}) ->
-    case dets:first(?TAB) of
-        '$end_of_table' ->
-            {reply, error, State};
-        Key ->
-            [Entry] = dets:lookup(?TAB, Key),
-            {reply, {ok, Entry}, State}
-    end;
-handle_call({delete, Id}, _From, State = #state{}) ->
-    ok = dets:delete(?TAB, Id),
-    {reply, ok, State};
-
-handle_call(get_all_cache, _From, State = #state{}) ->
-    Items = dets:foldl(fun(Record, Acc) -> [Record|Acc] end, [], ?TAB),
-    {reply, {ok, lists:reverse(Items)}, State};
-
 handle_call(_Request, _From, State = #state{}) ->
     {reply, ok, State}.
 
@@ -104,8 +73,9 @@ handle_call(_Request, _From, State = #state{}) ->
     {noreply, NewState :: #state{}} |
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast(_Request, State = #state{}) ->
-    {noreply, State}.
+handle_cast({monitor_container, ReceiverPid, ContainerId}, State = #state{monitors = Monitors}) ->
+    MRef = erlang:monitor(process, ReceiverPid),
+    {noreply, State#state{monitors = maps:put(ContainerId, {ReceiverPid, MRef}, Monitors)}}.
 
 %% @private
 %% @doc Handling all non call/cast messages
@@ -113,8 +83,31 @@ handle_cast(_Request, State = #state{}) ->
     {noreply, NewState :: #state{}} |
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_info(_Info, State = #state{}) ->
-    {noreply, State}.
+handle_info({timeout, _, attach_docker_events}, State = #state{port = undefined}) ->
+    ExecCmd = "docker events --format \"{{json .}}\"",
+    case catch erlang:open_port({spawn, ExecCmd}, [exit_status, {line, 10239}, use_stdio, stderr_to_stdout, binary]) of
+        Port when is_port(Port) ->
+            {noreply, State#state{port = Port}};
+        _Error ->
+            try_attach_events(5000),
+            {noreply, State}
+    end;
+handle_info({Port, {data, {eol, BinLine}}}, State = #state{port = Port}) ->
+    Event = catch json:decode(BinLine),
+    logger:debug("event: ~p", [Event]),
+    handle_event(Event, State),
+    {noreply, State};
+
+%% 进程退出的时候删除掉管理的Pid
+handle_info({'DOWN', MRef, process, _Pid, _Reason}, State = #state{monitors = Monitors}) ->
+    NMonitors = maps:filter(fun(_Key, {_, Ref}) -> MRef =/= Ref end, Monitors),
+    {noreply, State#state{monitors = NMonitors}};
+
+%% Port退出的时候，尝试重启
+handle_info({'EXIT', Port, Reason}, State = #state{port = Port}) ->
+    logger:warning("[efka_docker_events] exit with reason: ~p", [Reason]),
+    try_attach_events(5000),
+    {noreply, State#state{port = undefined}}.
 
 %% @private
 %% @doc This function is called by a gen_server when it is about to
@@ -124,7 +117,6 @@ handle_info(_Info, State = #state{}) ->
 -spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
     State :: #state{}) -> term()).
 terminate(_Reason, _State = #state{}) ->
-    dets:close(?TAB),
     ok.
 
 %% @private
@@ -138,7 +130,24 @@ code_change(_OldVsn, State = #state{}, _Extra) ->
 %%%===================================================================
 %%% Internal functions
 %%%===================================================================
+-spec handle_event(term(), #state{}) -> ok.
+handle_event(#{<<"Type">> := <<"container">>, <<"status">> := Status, <<"id">> := Id}, #state{monitors = Monitors}) ->
+    case maps:find(Id, Monitors) of
+        error ->
+            ok;
+        {ok, {ReceiverPid, _}} ->
+            case Status of
+                <<"start">> ->
+                    ReceiverPid ! {docker_events, start};
+                <<"stop">> ->
+                    ReceiverPid ! {docker_events, stop};
+                _ ->
+                    ok
+            end
+    end;
+handle_event(_, _) ->
+    ok.
 
--spec generate_id() -> integer().
-generate_id() ->
-    os:system_time(microsecond).
+-spec try_attach_events(non_neg_integer()) -> reference().
+try_attach_events(Timeout) ->
+    erlang:start_timer(Timeout, self(), attach_docker_events).

apps/docker/src/docker_helper.erl

@@ -0,0 +1,102 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 17. 9月 2025 14:50
+%%%-------------------------------------------------------------------
+-module(docker_helper).
+-author("anlicheng").
+
+%% API
+-export([root_dir/0]).
+-export([ensure_container_dir/2, ensure_container_dir/3, get_container_dir/2, get_config_file/1]).
+-export([update_container_config/2]).
+
+-spec root_dir() -> {ok, string()} | undefined.
+root_dir() ->
+    case application:get_env(docker, root_dir) of
+        {ok, RootDir} ->
+            {ok, RootDir};
+        undefined ->
+            application:get_env(efka, root_dir)
+    end.
+
+-spec ensure_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()}.
+ensure_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
+    ensure_container_dir(RootDir, ContainerName, <<>>).
+
+-spec ensure_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) ->
+    {ok, ServerRootDir :: string()}.
+ensure_container_dir(RootDir, ContainerName, ContainerDir) when is_list(RootDir), is_binary(ContainerName), is_binary(ContainerDir) ->
+    PointerDir = default_container_dir(RootDir, ContainerName),
+    PointerFile = container_dir_pointer_file(PointerDir),
+    ActualContainerDir = resolve_container_dir(RootDir, ContainerName, ContainerDir),
+    ok = filelib:ensure_dir(PointerDir),
+    ok = filelib:ensure_dir(ActualContainerDir),
+    ok = file:write_file(PointerFile, unicode:characters_to_binary(ActualContainerDir), [write]),
+    {ok, ActualContainerDir}.
+
+-spec get_config_file(ContainerDir :: string()) -> ConfigFile :: string().
+get_config_file(ContainerDir) when is_list(ContainerDir) ->
+    %% 根目录
+    ContainerDir ++ "service.conf".
+
+-spec get_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()} | error.
+get_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
+    ContainerRootDir = default_container_dir(RootDir, ContainerName),
+    PointerFile = container_dir_pointer_file(ContainerRootDir),
+    ResolvedContainerDir = case file:read_file(PointerFile) of
+        {ok, ActualContainerDirBin} ->
+            normalize_container_dir(binary_to_list(ActualContainerDirBin));
+        {error, _} ->
+            ContainerRootDir
+    end,
+    case filelib:is_dir(ResolvedContainerDir) of
+        true ->
+            {ok, ResolvedContainerDir};
+        false ->
+            error
+    end.
+
+-spec update_container_config(binary(), binary()) -> ok | {error, binary()}.
+update_container_config(ContainerName, Config) when is_binary(ContainerName), is_binary(Config) ->
+    {ok, RootDir} = root_dir(),
+    case get_container_dir(RootDir, ContainerName) of
+        {ok, ContainerDir} ->
+            ConfigFile = get_config_file(ContainerDir),
+            case file:write_file(ConfigFile, Config, [write, binary]) of
+                ok ->
+                    logger:warning("[efka_iot_client] write config file: ~p success", [ConfigFile]),
+                    ok;
+                {error, Reason} ->
+                    logger:warning("[efka_iot_client] write config file: ~p, get error: ~p", [ConfigFile, Reason]),
+                    {error, <<"write config failed">>}
+            end;
+        error ->
+            {error, <<"error">>}
+    end.
+
+-spec default_container_dir(RootDir :: string(), ContainerName :: binary()) -> string().
+default_container_dir(RootDir, ContainerName) ->
+    normalize_container_dir(RootDir ++ "/" ++ binary_to_list(ContainerName)).
+
+-spec container_dir_pointer_file(ContainerDir :: string()) -> string().
+container_dir_pointer_file(ContainerDir) ->
+    ContainerDir ++ ".container_dir".
+
+-spec resolve_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) -> string().
+resolve_container_dir(RootDir, ContainerName, <<>>) ->
+    default_container_dir(RootDir, ContainerName);
+resolve_container_dir(_RootDir, _ContainerName, ContainerDir) ->
+    normalize_container_dir(binary_to_list(ContainerDir)).
+
+-spec normalize_container_dir(ContainerDir :: string()) -> string().
+normalize_container_dir(ContainerDir) ->
+    case lists:last(ContainerDir) of
+        $/ ->
+            ContainerDir;
+        _ ->
+            ContainerDir ++ "/"
+    end.

apps/docker/src/docker_sup.erl

@@ -0,0 +1,42 @@
+%%%-------------------------------------------------------------------
+%% @doc Docker top level supervisor.
+%% @end
+%%%-------------------------------------------------------------------
+
+-module(docker_sup).
+
+-behaviour(supervisor).
+
+-export([start_link/0]).
+-export([init/1]).
+
+-define(SERVER, ?MODULE).
+
+-spec start_link() -> {ok, pid()} | ignore | {error, term()}.
+start_link() ->
+    supervisor:start_link({local, ?SERVER}, ?MODULE, []).
+
+-spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
+init([]) ->
+    SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
+    ChildSpecs = [
+        #{
+            id => docker_task_reporter,
+            start => {docker_task_reporter, start_link, []},
+            restart => permanent,
+            shutdown => 2000,
+            type => worker,
+            modules => [docker_task_reporter]
+        },
+
+        #{
+            id => docker_deploy_manager,
+            start => {docker_deploy_manager, start_link, []},
+            restart => permanent,
+            shutdown => 2000,
+            type => worker,
+            modules => [docker_deploy_manager]
+        }
+    ],
+
+    {ok, {SupFlags, ChildSpecs}}.

apps/docker/src/docker_task_reporter.erl

@@ -0,0 +1,121 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2026, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 20. 4月 2026
+%%%-------------------------------------------------------------------
+-module(docker_task_reporter).
+-author("anlicheng").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0]).
+-export([stream/3, close/2]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+-define(FLUSH_INTERVAL, 1000).
+
+-record(state, {
+    pending = queue:new(),
+    flush_ref = undefined
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec stream(TaskId :: integer(), Type :: binary(), Stream :: binary()) -> ok.
+stream(TaskId, Type, Stream) when is_integer(TaskId), is_binary(Type), is_binary(Stream) ->
+    gen_server:cast(?SERVER, {stream, TaskId, Type, Stream}).
+
+-spec close(TaskId :: integer(), Reason :: binary()) -> ok.
+close(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) ->
+    gen_server:cast(?SERVER, {close, TaskId, Reason}).
+
+-spec start_link() -> {ok, pid()} | ignore | {error, term()}.
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+-spec init(list()) -> {ok, #state{}}.
+init([]) ->
+    {ok, #state{}}.
+
+-spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, ok, #state{}}.
+handle_call(_Request, _From, State) ->
+    {reply, ok, State}.
+
+-spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
+handle_cast({stream, TaskId, Type, Stream}, State0 = #state{pending = Pending0}) ->
+    Pending = queue:in({stream, TaskId, Type, Stream}, Pending0),
+    {noreply, ensure_flush_timer(State0#state{pending = Pending}, 0)};
+handle_cast({close, TaskId, Reason}, State0 = #state{pending = Pending0}) ->
+    Pending = queue:in({close, TaskId, Reason}, Pending0),
+    {noreply, ensure_flush_timer(State0#state{pending = Pending}, 0)};
+handle_cast(_Request, State) ->
+    {noreply, State}.
+
+-spec handle_info(term(), #state{}) -> {noreply, #state{}}.
+handle_info(flush, State0 = #state{}) ->
+    State1 = State0#state{flush_ref = undefined},
+    {noreply, flush_pending(State1)};
+handle_info(_Info, State) ->
+    {noreply, State}.
+
+-spec terminate(term(), #state{}) -> ok.
+terminate(_Reason, _State) ->
+    ok.
+
+-spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
+code_change(_OldVsn, State, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec ensure_flush_timer(#state{}, non_neg_integer()) -> #state{}.
+ensure_flush_timer(State = #state{flush_ref = undefined}, Delay) when is_integer(Delay), Delay >= 0 ->
+    Ref = erlang:send_after(Delay, self(), flush),
+    State#state{flush_ref = Ref};
+ensure_flush_timer(State = #state{}, _Delay) ->
+    State.
+
+-spec flush_pending(#state{}) -> #state{}.
+flush_pending(State = #state{pending = Pending0}) ->
+    case queue:out(Pending0) of
+        {empty, _} ->
+            State;
+        {{value, Event}, Pending1} ->
+            case send_event(Event) of
+                ok ->
+                    flush_pending(State#state{pending = Pending1});
+                not_ready ->
+                    ensure_flush_timer(State#state{pending = Pending0}, ?FLUSH_INTERVAL)
+            end
+    end.
+
+-spec send_event({stream, integer(), binary(), binary()} | {close, integer(), binary()}) -> ok | not_ready.
+send_event({stream, TaskId, Type, Stream}) ->
+    maybe_send(fun() -> efka_iot_client:task_event_stream(TaskId, Type, Stream) end);
+send_event({close, TaskId, Reason}) ->
+    maybe_send(fun() -> efka_iot_client:close_task_event_stream(TaskId, Reason) end).
+
+-spec maybe_send(fun(() -> any())) -> ok | not_ready.
+maybe_send(SendFun) ->
+    case catch efka_iot_client:is_activated() of
+        true ->
+            _ = catch SendFun(),
+            ok;
+        _ ->
+            not_ready
+    end.

apps/docker/test/efka_docker_container_builder_tests.erl

@@ -0,0 +1,35 @@
+-module(efka_docker_container_builder_tests).
+
+-include_lib("eunit/include/eunit.hrl").
+
+port_bindings_are_rendered_for_docker_create_test() ->
+    Name = <<"nginx">>,
+    ContainerDir = "/tmp/efka-nginx/",
+    Create = #{
+        <<"config">> => #{
+            <<"image">> => <<"nginx:latest">>,
+            <<"exposed_ports">> => [
+                #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
+                #{<<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
+            ]
+        },
+        <<"host_config">> => #{
+            <<"port_bindings">> => [
+                #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
+                #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
+            ]
+        }
+    },
+    Options = docker_container_builder:build_options(Name, ContainerDir, Create),
+    ?assertMatch(#{
+        <<"ExposedPorts">> := #{
+            <<"80/tcp">> := #{},
+            <<"443/tcp">> := #{}
+        },
+        <<"HostConfig">> := #{
+            <<"PortBindings">> := #{
+                <<"80/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"8080">>}],
+                <<"443/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"443">>}]
+            }
+        }
+    }, Options).

apps/efka/include/efka.hrl

@@ -1,54 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%% 扩展部分, 1: 支持基于topic的pub/sub机制; 2. 基于target的单点通讯和广播
-%%% @end
-%%% Created : 21. 4月 2025 17:28
-%%%-------------------------------------------------------------------
--author("anlicheng").
-
-%% efka主动发起的消息体类型, 消息大类
--define(PACKET_REQUEST, 16#01).
--define(PACKET_RESPONSE, 16#02).
-
-%% 服务器基于pub/sub的消息, 消息大类
--define(PACKET_PUB, 16#03).
-
-%% push调用不需要返回, 消息大类
--define(PACKET_COMMAND, 16#04).
-
-%% 服务器端推送消息
--define(PACKET_ASYNC_CALL, 16#05).
--define(PACKET_ASYNC_CALL_REPLY, 16#06).
-
-%% ping包，客户端主动发起
--define(PACKET_PING, 16#FF).
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%%% 二级分类定义
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-%% 主机端上报数据类型标识
--define(METHOD_AUTH, 16#01).
--define(METHOD_DATA, 16#02).
--define(METHOD_PING, 16#03).
--define(METHOD_INFORM, 16#04).
--define(METHOD_EVENT, 16#05).
--define(METHOD_PHASE, 16#06).
--define(METHOD_REQUEST_SERVICE_CONFIG, 16#07).
-
-%%%% 命令类型子分类, 不需要返回值
-
-%% 授权
--define(COMMAND_AUTH, 16#08).
-
-%%%% 主动推送的消息类型子分类, 需要返回值
-
--define(PUSH_DEPLOY, 16#01).
--define(PUSH_START_SERVICE, 16#02).
--define(PUSH_STOP_SERVICE, 16#03).
-
--define(PUSH_SERVICE_CONFIG, 16#04).
--define(PUSH_INVOKE, 16#05).
--define(PUSH_TASK_LOG, 16#06).

apps/efka/include/efka_service_pb.hrl

@@ -0,0 +1,72 @@
+%% -*- coding: utf-8 -*-
+%% Automatically generated, do not edit
+%% Generated by gpb_compile version 4.21.7
+
+-ifndef(efka_service_pb).
+-define(efka_service_pb, true).
+
+-define(efka_service_pb_gpb_version, "4.21.7").
+
+
+-ifndef('SERVICEREQUEST.REGISTER_PB_H').
+-define('SERVICEREQUEST.REGISTER_PB_H', true).
+-record('ServiceRequest.Register',
+        {service_id = <<>>      :: unicode:chardata() | undefined % = 1, optional
+        }).
+-endif.
+
+-ifndef('SERVICEREQUEST.SUBSCRIBE_PB_H').
+-define('SERVICEREQUEST.SUBSCRIBE_PB_H', true).
+-record('ServiceRequest.Subscribe',
+        {topic = <<>>           :: unicode:chardata() | undefined % = 1, optional
+        }).
+-endif.
+
+-ifndef('SERVICEREQUEST_PB_H').
+-define('SERVICEREQUEST_PB_H', true).
+-record('ServiceRequest',
+        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 32 bits
+         request                :: {register, efka_service_pb:'ServiceRequest.Register'()} | {subscribe, efka_service_pb:'ServiceRequest.Subscribe'()} | undefined % oneof
+        }).
+-endif.
+
+-ifndef('SERVICEREPLY.ERROR_PB_H').
+-define('SERVICEREPLY.ERROR_PB_H', true).
+-record('ServiceReply.Error',
+        {code = 0               :: integer() | undefined, % = 1, optional, 32 bits
+         message = <<>>         :: unicode:chardata() | undefined % = 2, optional
+        }).
+-endif.
+
+-ifndef('SERVICEREPLY_PB_H').
+-define('SERVICEREPLY_PB_H', true).
+-record('ServiceReply',
+        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 32 bits
+         reply                  :: {result, iodata()} | {error, efka_service_pb:'ServiceReply.Error'()} | undefined % oneof
+        }).
+-endif.
+
+-ifndef('SERVICECAST.METRICDATA_PB_H').
+-define('SERVICECAST.METRICDATA_PB_H', true).
+-record('ServiceCast.MetricData',
+        {route_key = <<>>       :: iodata() | undefined, % = 1, optional
+         metric = <<>>          :: iodata() | undefined % = 2, optional
+        }).
+-endif.
+
+-ifndef('SERVICECAST.TOPICEVENT_PB_H').
+-define('SERVICECAST.TOPICEVENT_PB_H', true).
+-record('ServiceCast.TopicEvent',
+        {topic = <<>>           :: unicode:chardata() | undefined, % = 1, optional
+         content = <<>>         :: iodata() | undefined % = 2, optional
+        }).
+-endif.
+
+-ifndef('SERVICECAST_PB_H').
+-define('SERVICECAST_PB_H', true).
+-record('ServiceCast',
+        {body                   :: {topic_event, efka_service_pb:'ServiceCast.TopicEvent'()} | {metric_data, efka_service_pb:'ServiceCast.MetricData'()} | undefined % oneof
+        }).
+-endif.
+
+-endif.

apps/efka/include/efka_tables.hrl

@@ -4,37 +4,21 @@
 %%% @doc
 %%%
 %%% @end
-%%% Created : 30. 4月 2025 11:16
+%%% Created : 29. 9月 2025 15:27
 %%%-------------------------------------------------------------------
 -author("anlicheng").
 
+-define(SERVICE_STOPPED, 0).
+-define(SERVICE_RUNNING, 1).
+
 %% 用来保存微服务
 -record(service, {
     service_id :: binary(),
-    tar_url :: binary(),
-    %% 工作目录
-    root_dir :: string(),
-    %% 配置信息
-    config_json :: binary(),
+    container_name :: binary(),
+    %% 配置信息, 微服务启动的时候自己注册的信息
+    meta_data = #{} :: map(),
     %% 状态： 0: 停止, 1: 运行中
-    status = 0
-}).
-
-%% 数据缓存
--record(cache, {
-    id = 0 :: integer(),
-    method :: integer(),
-    data :: binary()
-}).
-
-%% 数据缓存
--record(task_log, {
-    task_id = 0 :: integer(),
-    logs = []:: list()
-}).
-
-%% id生成器
--record(id_generator, {
-    id,
-    value = 1
+    status = 0,
+    create_ts  = 0 :: integer(),
+    update_ts  = 0 :: integer()
 }).

apps/efka/include/message_pb.hrl

@@ -1,128 +0,0 @@
-%% -*- coding: utf-8 -*-
-%% Automatically generated, do not edit
-%% Generated by gpb_compile version 4.21.1
-
--ifndef(message_pb).
--define(message_pb, true).
-
--define(message_pb_gpb_version, "4.21.1").
-
-
--ifndef('AUTH_REQUEST_PB_H').
--define('AUTH_REQUEST_PB_H', true).
--record(auth_request,
-        {uuid = <<>>            :: unicode:chardata() | undefined, % = 1, optional
-         username = <<>>        :: unicode:chardata() | undefined, % = 2, optional
-         salt = <<>>            :: unicode:chardata() | undefined, % = 4, optional
-         token = <<>>           :: unicode:chardata() | undefined, % = 5, optional
-         timestamp = 0          :: non_neg_integer() | undefined % = 6, optional, 32 bits
-        }).
--endif.
-
--ifndef('AUTH_REPLY_PB_H').
--define('AUTH_REPLY_PB_H', true).
--record(auth_reply,
-        {code = 0               :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         message = <<>>         :: unicode:chardata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('PUB_PB_H').
--define('PUB_PB_H', true).
--record(pub,
-        {topic = <<>>           :: unicode:chardata() | undefined, % = 1, optional
-         content = <<>>         :: unicode:chardata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('ASYNC_CALL_REPLY_PB_H').
--define('ASYNC_CALL_REPLY_PB_H', true).
--record(async_call_reply,
-        {code = 0               :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         result = <<>>          :: unicode:chardata() | undefined, % = 2, optional
-         message = <<>>         :: unicode:chardata() | undefined % = 3, optional
-        }).
--endif.
-
--ifndef('DEPLOY_PB_H').
--define('DEPLOY_PB_H', true).
--record(deploy,
-        {task_id = 0            :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         service_id = <<>>      :: unicode:chardata() | undefined, % = 2, optional
-         tar_url = <<>>         :: unicode:chardata() | undefined % = 3, optional
-        }).
--endif.
-
--ifndef('FETCH_TASK_LOG_PB_H').
--define('FETCH_TASK_LOG_PB_H', true).
--record(fetch_task_log,
-        {task_id = 0            :: non_neg_integer() | undefined % = 1, optional, 32 bits
-        }).
--endif.
-
--ifndef('INVOKE_PB_H').
--define('INVOKE_PB_H', true).
--record(invoke,
-        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-         payload = <<>>         :: unicode:chardata() | undefined, % = 2, optional
-         timeout = 0            :: non_neg_integer() | undefined % = 3, optional, 32 bits
-        }).
--endif.
-
--ifndef('PUSH_SERVICE_CONFIG_PB_H').
--define('PUSH_SERVICE_CONFIG_PB_H', true).
--record(push_service_config,
-        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-         config_json = <<>>     :: unicode:chardata() | undefined, % = 2, optional
-         timeout = 0            :: non_neg_integer() | undefined % = 3, optional, 32 bits
-        }).
--endif.
-
--ifndef('DATA_PB_H').
--define('DATA_PB_H', true).
--record(data,
-        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-         device_uuid = <<>>     :: unicode:chardata() | undefined, % = 2, optional
-         metric = <<>>          :: unicode:chardata() | undefined % = 3, optional
-        }).
--endif.
-
--ifndef('PING_PB_H').
--define('PING_PB_H', true).
--record(ping,
-        {adcode = <<>>          :: unicode:chardata() | undefined, % = 1, optional
-         boot_time = 0          :: non_neg_integer() | undefined, % = 2, optional, 32 bits
-         province = <<>>        :: unicode:chardata() | undefined, % = 3, optional
-         city = <<>>            :: unicode:chardata() | undefined, % = 4, optional
-         efka_version = <<>>    :: unicode:chardata() | undefined, % = 5, optional
-         kernel_arch = <<>>     :: unicode:chardata() | undefined, % = 6, optional
-         ips = []               :: [unicode:chardata()] | undefined, % = 7, repeated
-         cpu_core = 0           :: non_neg_integer() | undefined, % = 8, optional, 32 bits
-         cpu_load = 0           :: non_neg_integer() | undefined, % = 9, optional, 32 bits
-         cpu_temperature = 0.0  :: float() | integer() | infinity | '-infinity' | nan | undefined, % = 10, optional
-         disk = []              :: [integer()] | undefined, % = 11, repeated, 32 bits
-         memory = []            :: [integer()] | undefined, % = 12, repeated, 32 bits
-         interfaces = <<>>      :: unicode:chardata() | undefined % = 13, optional
-        }).
--endif.
-
--ifndef('SERVICE_INFORM_PB_H').
--define('SERVICE_INFORM_PB_H', true).
--record(service_inform,
-        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-         props = <<>>           :: unicode:chardata() | undefined, % = 2, optional
-         status = 0             :: non_neg_integer() | undefined, % = 3, optional, 32 bits
-         timestamp = 0          :: non_neg_integer() | undefined % = 4, optional, 32 bits
-        }).
--endif.
-
--ifndef('EVENT_PB_H').
--define('EVENT_PB_H', true).
--record(event,
-        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-         event_type = 0         :: non_neg_integer() | undefined, % = 2, optional, 32 bits
-         params = <<>>          :: unicode:chardata() | undefined % = 3, optional
-        }).
--endif.
-
--endif.

apps/efka/proto/service.proto

@@ -0,0 +1,54 @@
+syntax = "proto3";
+
+// 微服务通过 ws_channel 与 efka 之间的通道消息定义。
+// 该协议用于替代当前基于 JSON 的请求/响应/推送格式。
+
+message ServiceRequest {
+  uint32 packet_id = 1;
+
+  message Register {
+    string service_id = 1;
+  }
+
+  message Subscribe {
+    string topic = 1;
+  }
+
+  oneof request {
+    Register register = 10;
+    Subscribe subscribe = 11;
+  }
+}
+
+message ServiceReply {
+  uint32 packet_id = 1;
+
+  message Error {
+    int32 code = 1;
+    string message = 2;
+  }
+
+  oneof reply {
+    bytes result = 10;
+    Error error = 11;
+  }
+}
+
+message ServiceCast {
+
+  message MetricData {
+    bytes route_key = 1;
+    bytes metric = 2;
+  }
+
+  message TopicEvent {
+    string topic = 1;
+    bytes content = 2;
+  }
+
+  oneof body {
+    TopicEvent topic_event = 10;
+    MetricData metric_data = 11;
+  }
+
+}

apps/efka/rebar.config

@@ -0,0 +1,23 @@
+{erl_opts, [{i, "include"}]}.
+
+{gpb_opts, [
+    {i, "proto"},
+    {f, ["service.proto"]},
+    recursive,
+    {module_name_prefix, "efka_"},
+    {module_name_suffix, "_pb"},
+    {o_erl, "src/protobuf"},
+    {o_hrl, "include"},
+    include_as_lib,
+    {strings_as_binaries, true},
+    type_specs,
+    report,
+    verbose
+]}.
+
+{provider_hooks, [
+    {pre, [
+        {compile, {protobuf, compile}},
+        {clean, {protobuf, clean}}
+    ]}
+]}.

apps/efka/src/efka.app.src

@@ -6,20 +6,14 @@
   {applications,
    [
     sync,
-    jiffy,
-    %gpb,
-    mnesia,
-    parse_trans,
-    lager,
+    docker,
+    cowboy,
+    ranch,
     crypto,
+    cowlib,
     inets,
     ssl,
     public_key,
-
-    %erts,
-    %runtime_tools,
-    %observer,
-
     kernel,
     stdlib
    ]},

apps/efka/src/efka_agent.erl

@@ -1,435 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 21. 5月 2025 18:38
-%%%-------------------------------------------------------------------
--module(efka_agent).
--author("anlicheng").
--include("message_pb.hrl").
--include("efka.hrl").
--include("efka_tables.hrl").
-
--behaviour(gen_statem).
-
-%% API
--export([start_link/0]).
--export([metric_data/3, event/3, ping/13, request_service_config/2, await_reply/2]).
-
-%% gen_statem callbacks
--export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]).
-
--define(SERVER, ?MODULE).
-
-%% 标记当前agent的状态，只有在 activated 状态下才可以正常的发送数据
--define(STATE_DENIED, denied).
--define(STATE_CONNECTING, connecting).
--define(STATE_AUTH, auth).
-%% 不能推送消息到服务，但是可以接受服务器的部分指令
--define(STATE_RESTRICTED, restricted).
-%% 激活状态下
--define(STATE_ACTIVATED, activated).
-
--record(state, {
-    transport_pid :: undefined | pid(),
-    transport_ref :: undefined | reference(),
-    %% 服务器端推送的消息的未确认列表, 映射关系 #{Ref => PacketId}
-    push_inflight = #{},
-    %% 发送的请求的未确认列表, 映射关系 #{Ref => ReceiverPid}
-    request_inflight = #{}
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
-%% 发送数据
--spec metric_data(ServiceId :: binary(), DeviceUUID::binary(), LineProtocolData :: binary()) -> no_return().
-metric_data(ServiceId, DeviceUUID, LineProtocolData) when is_binary(ServiceId), is_binary(DeviceUUID), is_binary(LineProtocolData) ->
-    gen_statem:cast(?SERVER, {metric_data, ServiceId, DeviceUUID, LineProtocolData}).
-
--spec event(ServiceId :: binary(), EventType :: integer(), Params :: binary()) -> no_return().
-event(ServiceId, EventType, Params) when is_binary(ServiceId), is_integer(EventType), is_binary(Params) ->
-    gen_statem:cast(?SERVER, {event, ServiceId, EventType, Params}).
-
-ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) ->
-    gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}).
-
-%% 请求微服务的配置
--spec request_service_config(ReceiverPid :: pid(), ServiceId :: binary()) -> {ok, Ref :: reference()} | {error, Reason :: term()}.
-request_service_config(ReceiverPid, ServiceId) when is_binary(ServiceId) ->
-    gen_statem:call(?SERVER, {request_service_config, ReceiverPid, ServiceId}).
-
-%% 等待消息的回复
--spec await_reply(Ref :: reference(), Timeout :: timeout()) -> {ok, Reply :: binary()} | {error, timeout}.
-await_reply(Ref, Timeout) when is_reference(Ref), is_integer(Timeout) ->
-    receive
-        {request_reply, Ref, ReplyBin} ->
-            {ok, ReplyBin}
-    after Timeout ->
-        {error, timeout}
-    end.
-
-%% @doc Creates a gen_statem process which calls Module:init/1 to
-%% initialize. To ensure a synchronized start-up procedure, this
-%% function does not return until Module:init/1 has returned.
-start_link() ->
-    gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []).
-
-%%%===================================================================
-%%% gen_statem callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Whenever a gen_statem is started using gen_statem:start/[3,4] or
-%% gen_statem:start_link/[3,4], this function is called by the new
-%% process to initialize.
-init([]) ->
-    erlang:start_timer(0, self(), create_transport),
-    {ok, ?STATE_DENIED, #state{}}.
-
-%% @private
-%% @doc This function is called by a gen_statem when it needs to find out
-%% the callback mode of the callback module.
-callback_mode() ->
-    handle_event_function.
-
-%% @private
-%% @doc If callback_mode is handle_event_function, then whenever a
-%% gen_statem receives an event from call/2, cast/2, or as a normal
-%% process message, this function is called.
-handle_event({call, From}, {request_service_config, ReceiverPid, ServiceId}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid, request_inflight = RequestInflight}) ->
-    Ref = efka_transport:request(TransportPid, ?METHOD_REQUEST_SERVICE_CONFIG, ServiceId),
-    {keep_state, State#state{request_inflight = maps:put(Ref, ReceiverPid, RequestInflight)}, [{reply, From, {ok, Ref}}]};
-
-handle_event({call, From}, {request_service_config, _ReceiverPid, _ServiceId}, _, State) ->
-    {keep_state, State, [{reply, From, {error, <<"transport is not alive">>}}]};
-
-%% 异步发送数据, 连接存在时候直接发送；否则缓存到mnesia
-handle_event(cast, {metric_data, ServiceId, DeviceUUID, LineProtocolData}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
-    Packet = message_pb:encode_msg(#data{
-        service_id = ServiceId,
-        device_uuid = DeviceUUID,
-        metric = LineProtocolData
-    }),
-    efka_transport:send(TransportPid, ?METHOD_DATA, Packet),
-    {keep_state, State};
-
-handle_event(cast, {metric_data, ServiceId, DeviceUUID, LineProtocolData}, _, State) ->
-    Packet = message_pb:encode_msg(#data{
-        service_id = ServiceId,
-        device_uuid = DeviceUUID,
-        metric = LineProtocolData
-    }),
-    ok = cache_model:insert(?METHOD_DATA, Packet),
-    {keep_state, State};
-
-%% 异步发送事件
-handle_event(cast, {event, ServiceId, EventType, Params}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
-    EventPacket = message_pb:encode_msg(#event{
-        service_id = ServiceId,
-        event_type = EventType,
-        params = Params
-    }),
-    efka_transport:send(TransportPid, ?METHOD_EVENT, EventPacket),
-    {keep_state, State};
-handle_event(cast, {event, ServiceId, EventType, Params}, ?STATE_ACTIVATED, State) ->
-    EventPacket = message_pb:encode_msg(#event{
-        service_id = ServiceId,
-        event_type = EventType,
-        params = Params
-    }),
-    ok = cache_model:insert(?METHOD_EVENT, EventPacket),
-    {keep_state, State};
-
-handle_event(cast, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}, ?STATE_ACTIVATED,
-    State = #state{transport_pid = TransportPid}) ->
-
-    Ping = message_pb:encode_msg(#ping{
-        adcode = AdCode,
-        boot_time = BootTime,
-        province = Province,
-        city = City,
-        efka_version = EfkaVersion,
-        kernel_arch = KernelArch,
-        ips = Ips,
-        cpu_core = CpuCore,
-        cpu_load = CpuLoad,
-        cpu_temperature = CpuTemperature,
-        disk = Disk,
-        memory = Memory,
-        interfaces = Interfaces
-    }),
-    efka_transport:send(TransportPid, ?METHOD_PING, Ping),
-    {keep_state, State};
-
-%% 异步建立到服务器的连接
-handle_event(info, {timeout, _, create_transport}, ?STATE_DENIED, State) ->
-    {ok, Props} = application:get_env(efka, tls_server),
-    Host = proplists:get_value(host, Props),
-    Port = proplists:get_value(port, Props),
-    {ok, {TransportPid, TransportRef}} = efka_transport:start_monitor(self(), Host, Port),
-    efka_transport:connect(TransportPid),
-
-    {next_state, ?STATE_CONNECTING, State#state{transport_pid = TransportPid, transport_ref = TransportRef}};
-
-handle_event(info, {connect_reply, Reply}, ?STATE_CONNECTING, State = #state{transport_pid = TransportPid}) ->
-    case Reply of
-        ok ->
-            AuthBin = auth_request(),
-            efka_transport:auth_request(TransportPid, AuthBin),
-            {next_state, ?STATE_AUTH, State};
-        {error, Reason} ->
-            lager:debug("[efka_agent] connect failed, error: ~p, pid: ~p", [Reason, TransportPid]),
-            efka_transport:stop(TransportPid),
-            {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
-    end;
-
-handle_event(info, {auth_reply, Reply}, ?STATE_AUTH, State = #state{transport_pid = TransportPid}) ->
-    case Reply of
-        {ok, ReplyBin} ->
-            #auth_reply{code = Code, message = Message} = message_pb:decode_msg(ReplyBin, auth_reply),
-            case Code of
-                0 ->
-                    lager:debug("[efka_agent] auth success, message: ~p", [Message]),
-                    {next_state, ?STATE_ACTIVATED, State, [{next_event, info, flush_cache}]};
-                1 ->
-                    %% 主机在后台的授权未通过；此时agent不能推送数据给云端服务器，但是云端服务器可以推送命令给agent
-                    %% socket的连接状态需要维持
-                    lager:debug("[efka_agent] auth denied, message: ~p", [Message]),
-                    {next_state, ?STATE_RESTRICTED, State};
-                2 ->
-                    % 其他类型的错误，需要间隔时间重试
-                    lager:debug("[efka_agent] auth failed, message: ~p", [Message]),
-                    efka_transport:stop(TransportPid),
-                    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}};
-                _ ->
-                    % 其他类型的错误，需要间隔时间重试
-                    lager:debug("[efka_agent] auth failed, invalid message"),
-                    efka_transport:stop(TransportPid),
-                    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
-            end;
-        {error, Reason} ->
-            lager:debug("[efka_agent] auth_request failed, error: ~p", [Reason]),
-            efka_transport:stop(TransportPid),
-            {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
-    end;
-
-%% 将缓存中的数据推送到服务器端
-handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
-    case cache_model:fetch_next() of
-        {ok, #cache{id = Id, method = Method, data = Packet}} ->
-            efka_transport:send(TransportPid, Method, Packet),
-            cache_model:delete(Id),
-            {keep_state, State, [{next_event, info, flush_cache}]};
-        error ->
-            {keep_state, State}
-    end;
-handle_event(info, flush_cache, _, State) ->
-    {keep_state, State};
-
-%% 云端服务器推送了消息
-%% 激活消息
-
-%% 微服务部署
-handle_event(info, {server_async_call, PacketId, <<?PUSH_DEPLOY:8, DeployBin/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
-    #deploy{task_id = TaskId, service_id = ServiceId, tar_url = TarUrl} = message_pb:decode_msg(DeployBin, deploy),
-
-    %% 短暂的等待，efka_inetd收到消息后就立即返回了
-    Reply = case efka_inetd:deploy(TaskId, ServiceId, TarUrl) of
-                ok ->
-                    #async_call_reply{code = 1, result = <<"ok">>};
-                {error, Reason} when is_binary(Reason) ->
-                    #async_call_reply{code = 0, message = Reason}
-            end,
-    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-
-    {keep_state, State};
-
-%% 启动微服务
-handle_event(info, {server_async_call, PacketId, <<?PUSH_START_SERVICE:8, ServiceId/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
-    %% 短暂的等待，efka_inetd收到消息后就立即返回了
-    Reply = case efka_inetd:start_service(ServiceId) of
-                ok ->
-                    #async_call_reply{code = 1, result = <<"ok">>};
-                {error, Reason} when is_binary(Reason) ->
-                    #async_call_reply{code = 0, message = Reason}
-            end,
-    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-
-    {keep_state, State};
-
-%% 停止微服务
-handle_event(info, {server_async_call, PacketId, <<?PUSH_STOP_SERVICE:8, ServiceId/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
-    %% 短暂的等待，efka_inetd收到消息后就立即返回了
-    Reply = case efka_inetd:stop_service(ServiceId) of
-                ok ->
-                    #async_call_reply{code = 1, result = <<"ok">>};
-                {error, Reason} when is_binary(Reason) ->
-                    #async_call_reply{code = 0, message = Reason}
-            end,
-    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-
-    {keep_state, State};
-
-%% config.json配置信息
-handle_event(info, {server_async_call, PacketId, <<?PUSH_SERVICE_CONFIG:8, ConfigBin/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid, push_inflight = PushInflight}) ->
-    #push_service_config{service_id = ServiceId, config_json = ConfigJson, timeout = Timeout} = message_pb:decode_msg(ConfigBin, push_service_config),
-
-    case efka_service:get_pid(ServiceId) of
-        undefined ->
-            Reply = #async_call_reply{code = 0, message = <<"service not run">>},
-            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-            {keep_state, State};
-        ServicePid when is_pid(ServicePid) ->
-            Ref = make_ref(),
-            %% 将配置文件推送到对应的微服务
-            efka_service:push_config(ServicePid, Ref, ConfigJson),
-            %% 处理超时逻辑
-            erlang:start_timer(Timeout, self(), {request_timeout, Ref}),
-
-            {keep_state, State#state{push_inflight = maps:put(Ref, PacketId, PushInflight)}}
-    end;
-
-%% 收到需要回复的指令
-handle_event(info, {server_async_call, PacketId, <<?PUSH_INVOKE:8, InvokeBin/binary>>}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) ->
-    #invoke{service_id = ServiceId, payload = Payload, timeout = Timeout} = message_pb:decode_msg(InvokeBin, invoke),
-    %% 消息发送到订阅系统
-    case efka_service:get_pid(ServiceId) of
-        undefined ->
-            Reply = #async_call_reply{code = 0, message = <<"micro_service not run">>},
-            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-
-            {keep_state, State};
-        ServicePid when is_pid(ServicePid) ->
-            Ref = make_ref(),
-            efka_service:invoke(ServicePid, Ref, Payload),
-            %% 处理超时逻辑
-            erlang:start_timer(Timeout, self(), {request_timeout, Ref}),
-
-            {keep_state, State#state{push_inflight = maps:put(Ref, PacketId, PushInflight)}}
-    end;
-
-%% 处理task_log
-handle_event(info, {server_async_call, PacketId, <<?PUSH_TASK_LOG:8, TaskLogBin/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
-    #fetch_task_log{task_id = TaskId} = message_pb:decode_msg(TaskLogBin, fetch_task_log),
-    lager:debug("[efka_agent] get task_log request: ~p", [TaskId]),
-    {ok, Logs} = efka_inetd_task_log:get_logs(TaskId),
-    Reply = case length(Logs) > 0 of
-                true ->
-                    Result = iolist_to_binary(jiffy:encode(Logs, [force_utf8])),
-                    #async_call_reply{code = 1, result = Result};
-                false ->
-                    #async_call_reply{code = 1, result = <<"[]">>}
-            end,
-    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-
-    {keep_state, State};
-
-%% 处理命令
-handle_event(info, {server_command, ?COMMAND_AUTH, <<Auth:8>>}, StateName, State = #state{transport_pid = TransportPid}) ->
-    case {Auth, StateName} of
-        {1, ?STATE_ACTIVATED} ->
-            {keep_state, State};
-        {1, ?STATE_DENIED} ->
-            %% 重新激活, 需要重新校验
-            AuthRequestBin = auth_request(),
-            efka_transport:auth_request(TransportPid, AuthRequestBin),
-            {next_state, ?STATE_AUTH, State};
-        {0, _} ->
-            %% 这个时候的主机应该是受限制的状态，不允许发送消息；但是能够接受服务器推送的消息
-            {next_state, ?STATE_RESTRICTED, State}
-    end;
-
-%% 处理Pub/Sub机制
-handle_event(info, {server_pub, Topic, Content}, ?STATE_ACTIVATED, State) ->
-    lager:debug("[efka_agent] get pub topic: ~p, content: ~p", [Topic, Content]),
-    %% 消息发送到订阅系统
-    efka_subscription:publish(Topic, Content),
-    {keep_state, State};
-
-%% 收到来自efka_service的回复
-handle_event(info, {service_reply, Ref, EmsReply}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) ->
-    case maps:take(Ref, PushInflight) of
-        error ->
-            {keep_state, State};
-        {PacketId, NPushInflight} ->
-            Reply = case EmsReply of
-                        {ok, Result} ->
-                            #async_call_reply{code = 1, result = Result};
-                        {error, Reason} ->
-                            #async_call_reply{code = 0, message = Reason}
-                    end,
-            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-
-            {keep_state, State#state{push_inflight = NPushInflight}}
-    end;
-
-%% 收到来自服务器端的回复
-handle_event(info, {server_reply, Ref, ReplyBin}, ?STATE_ACTIVATED, State = #state{request_inflight = RequestInflight}) ->
-    case maps:take(Ref, RequestInflight) of
-        error ->
-            {keep_state, State};
-        {ReceiverPid, NRequestInflight} ->
-            is_process_alive(ReceiverPid) andalso erlang:send(ReceiverPid, {request_reply, Ref, ReplyBin}),
-            {keep_state, State#state{push_inflight = NRequestInflight}}
-    end;
-
-%% todo 请求超时逻辑处理
-handle_event(info, {timeout, _, {request_timeout, Ref}}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) ->
-    case maps:take(Ref, PushInflight) of
-        error ->
-            {keep_state, State};
-        {PacketId, NPushInflight} ->
-            Reply = #async_call_reply{code = 0, message = <<"reqeust timeout">>, result = <<>>},
-            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
-
-            {keep_state, State#state{push_inflight = NPushInflight}}
-    end;
-
-%% transport进程退出
-handle_event(info, {'DOWN', MRef, process, TransportPid, Reason}, _, State = #state{transport_ref = MRef}) ->
-    lager:debug("[efka_agent] transport pid: ~p, exit with reason: ~p", [TransportPid, Reason]),
-    erlang:start_timer(5000, self(), create_transport),
-    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined, transport_ref = undefined}}.
-
-%% @private
-%% @doc This function is called by a gen_statem when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_statem terminates with
-%% Reason. The return value is ignored.
-terminate(_Reason, _StateName, _State = #state{transport_pid = TransportPid}) ->
-    case is_pid(TransportPid) andalso is_process_alive(TransportPid) of
-        true ->
-            efka_transport:stop(TransportPid);
-        false ->
-            ok
-    end,
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
-code_change(_OldVsn, StateName, State = #state{}, _Extra) ->
-    {ok, StateName, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec auth_request() -> binary().
-auth_request() ->
-    {ok, AuthInfo} = application:get_env(efka, auth),
-    UUID = proplists:get_value(uuid, AuthInfo),
-    Username = proplists:get_value(username, AuthInfo),
-    Salt = proplists:get_value(salt, AuthInfo),
-    Token = proplists:get_value(token, AuthInfo),
-
-    message_pb:encode_msg(#auth_request{
-        uuid = unicode:characters_to_binary(UUID),
-        username = unicode:characters_to_binary(Username),
-        salt = unicode:characters_to_binary(Salt),
-        token = unicode:characters_to_binary(Token),
-        timestamp = efka_util:timestamp()
-    }).

apps/efka/src/efka_app.erl

@@ -9,46 +9,38 @@
 
 -export([start/2, stop/1]).
 
+-spec start(term(), term()) -> {ok, pid()} | {error, term()}.
 start(_StartType, _StartArgs) ->
     io:setopts([{encoding, unicode}]),
-    %% 启动mnesia数据库
-    start_mnesia(),
     %% 加速内存的回收
     erlang:system_flag(fullsweep_after, 16),
+    start_websocket_server(),
+
     efka_sup:start_link().
 
+-spec stop(term()) -> ok.
 stop(_State) ->
     ok.
 
-%% internal functions
+%% 微服务和efka之间通过websocket协议通讯
+-spec start_websocket_server() -> ok.
+start_websocket_server() ->
+    {ok, Props} = application:get_env(efka, websocket_server),
+    Acceptors = proplists:get_value(acceptors, Props, 50),
+    MaxConnections = proplists:get_value(max_connections, Props, 10240),
+    Backlog = proplists:get_value(backlog, Props, 1024),
+    Port = proplists:get_value(port, Props),
 
-%% 启动内存数据库
-start_mnesia() ->
-    %% 启动数据库
-    ensure_mnesia_schema(),
-    ok = mnesia:start(),
-    Tables = mnesia:system_info(tables),
-    lager:debug("[efka_app] tables: ~p", [Tables]),
-    %% 创建数据库表
-    not lists:member(id_generator, Tables) andalso id_generator_model:create_table(),
-    not lists:member(service, Tables) andalso service_model:create_table(),
-    not lists:member(cache, Tables) andalso cache_model:create_table(),
-    not lists:member(task_log, Tables) andalso task_log_model:create_table(),
-    ok.
+    Dispatcher = cowboy_router:compile([
+        {'_', [{"/ws", efka_service_channel, []}]}
+    ]),
 
--spec ensure_mnesia_schema() -> any().
-ensure_mnesia_schema() ->
-    case mnesia:system_info(use_dir) of
-        true ->
-            lager:debug("[efka_app] mnesia schema exists"),
-            ok;
-        false ->
-            mnesia:stop(),
-            case mnesia:create_schema([node()]) of
-                ok -> ok;
-                {error, {_, {already_exists, _}}} -> ok;
-                Error ->
-                    lager:debug("[iot_app] create mnesia schema failed with error: ~p", [Error]),
-                    throw({init_schema, Error})
-            end
-    end.
+    TransOpts = [
+        {port, Port},
+        {num_acceptors, Acceptors},
+        {backlog, Backlog},
+        {max_connections, MaxConnections}
+    ],
+    {ok, Pid} = cowboy:start_clear(ws_listener, TransOpts, #{env => #{dispatch => Dispatcher}}),
+
+    logger:debug("[efka_app] websocket server start at: ~p, pid is: ~p", [Port, Pid]).

apps/efka/src/efka_inetd.erl

@@ -1,212 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%% 微服务守护进程
-%%% 1. 负责微服务的下载, 版本的管理
-%%% 2. 目录管理等
-%%% @end
-%%% Created : 19. 4月 2025 14:55
-%%%-------------------------------------------------------------------
--module(efka_inetd).
--author("anlicheng").
--include("efka_tables.hrl").
--include("message_pb.hrl").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/0]).
--export([deploy/3, start_service/1, stop_service/1]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE).
-
--record(state, {
-    root_dir :: string(),
-    %% 建立任务到ref之间的映射, #{TaskPid => {TaskId, ServiceId}}
-    task_map = #{}
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec deploy(TaskId :: integer(), ServerId :: binary(), TarUrl :: binary()) -> ok | {error, Reason :: binary()}.
-deploy(TaskId, ServerId, TarUrl) when is_integer(TaskId), is_binary(ServerId), is_binary(TarUrl) ->
-    gen_server:call(?SERVER, {deploy, TaskId, ServerId, TarUrl}).
-
--spec start_service(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
-start_service(ServiceId) when is_binary(ServiceId) ->
-    gen_server:call(?SERVER, {start_service, ServiceId}).
-
--spec stop_service(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
-stop_service(ServiceId) when is_binary(ServiceId) ->
-    gen_server:call(?SERVER, {stop_service, ServiceId}).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link() ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link() ->
-    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([]) ->
-    erlang:process_flag(trap_exit, true),
-    {ok, RootDir} = application:get_env(efka, root_dir),
-    {ok, #state{root_dir = RootDir}}.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_call({deploy, TaskId, ServiceId, TarUrl}, _From, State = #state{root_dir = RootDir, task_map = TaskMap}) ->
-    %% 创建目录
-    {ok, ServiceRootDir} = ensure_dirs(RootDir, ServiceId),
-
-    ServicePid = efka_service:get_pid(ServiceId),
-    case is_pid(ServicePid) of
-        true ->
-            {reply, {error, <<"the service is running, stop first">>}, State};
-        false ->
-            case check_download_url(TarUrl) of
-                ok ->
-                    {ok, TaskPid} = efka_inetd_task:start_link(TaskId, ServiceRootDir, ServiceId, TarUrl),
-                    efka_inetd_task:deploy(TaskPid),
-                    lager:debug("[efka_inetd] start task_id: ~p, tar_url: ~p", [TaskId, TarUrl]),
-
-                    {reply, ok, State#state{task_map = maps:put(TaskPid, {TaskId, ServiceId}, TaskMap)}};
-                {error, Reason} ->
-                    lager:debug("[efka_inetd] check_download_url: ~p, get error: ~p", [TarUrl, Reason]),
-                    {reply, {error, <<"download url error">>}, State}
-            end
-    end;
-
-%% 启动服务: 当前服务如果正常运行，则不允许重启
-handle_call({start_service, ServiceId}, _From, State) ->
-    case efka_service:get_pid(ServiceId) of
-        undefined ->
-            case efka_service_sup:start_service(ServiceId) of
-                {ok, _} ->
-                    %% 更新数据库状态, 状态是为了保证下次efka重启的时候，服务能够启动
-                    ok = service_model:change_status(ServiceId, 1),
-                    {reply, ok, State};
-                {error, Reason} ->
-                    {reply, {error, Reason}, State}
-            end;
-        ServicePid when is_pid(ServicePid) ->
-            {reply, {error, <<"service is running">>}, State}
-    end;
-
-%% 停止服务, 主动停止的时候会改变服务配置的status字段
-handle_call({stop_service, ServiceId}, _From, State = #state{}) ->
-    case efka_service:get_pid(ServiceId) of
-        undefined ->
-            {reply, {error, <<"service not running">>}, State};
-        ServicePid when is_pid(ServicePid) ->
-            efka_service_sup:stop_service(ServiceId),
-            %% 主动停止的服务，需要更新数据库状态, 状态是为了保证下次efka重启的时候，不自动启动服务
-            ok = service_model:change_status(ServiceId, 0),
-            {reply, ok, State}
-    end;
-
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast(_Request, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_info({'EXIT', TaskPid, Reason}, State = #state{task_map = TaskMap}) ->
-    case maps:take(TaskPid, TaskMap) of
-        error ->
-            {noreply, State};
-        {{TaskId, ServiceId}, NTaskMap} ->
-            case Reason of
-                normal ->
-                    lager:debug("[efka_inetd] service_id: ~p, task_pid: ~p, exit normal", [ServiceId, TaskPid]),
-                    efka_inetd_task_log:flush(TaskId);
-                Error ->
-                    lager:notice("[efka_inetd] service_id: ~p, task_pid: ~p, exit with error: ~p", [ServiceId, TaskPid, Error]),
-                    efka_inetd_task_log:stash(TaskId, <<"task aborted">>),
-                    efka_inetd_task_log:flush(TaskId)
-            end,
-            {noreply, State#state{task_map = NTaskMap}}
-    end;
-
-handle_info(_Info, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(_Reason, _State = #state{}) ->
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec ensure_dirs(RootDir :: string(), ServerId :: binary()) -> {ok, ServerRootDir :: string()}.
-ensure_dirs(RootDir, ServerId) when is_list(RootDir), is_binary(ServerId) ->
-    %% 根目录
-    ServiceRootDir = RootDir ++ "/" ++ binary_to_list(ServerId) ++ "/",
-    ok = filelib:ensure_dir(ServiceRootDir),
-    {ok, ServiceRootDir}.
-
-%% 通过head请求先判定下载地址是否正确
--spec check_download_url(Url :: string() | binary()) -> ok | {error, Reason :: term()}.
-check_download_url(Url) when is_binary(Url) ->
-    check_download_url(binary_to_list(Url));
-check_download_url(Url) when is_list(Url) ->
-    SslOpts = [
-        {ssl, [
-            % 完全禁用证书验证
-            {verify, verify_none}
-        ]}
-    ],
-    case httpc:request(head, {Url, []}, SslOpts, [{sync, true}]) of
-        {ok, {{_, 200, "OK"}, _Headers, _}} ->
-            ok;
-        {error, Reason} ->
-            {error, Reason}
-    end.

apps/efka/src/efka_inetd_task.erl

@@ -1,241 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 07. 5月 2025 15:47
-%%%-------------------------------------------------------------------
--module(efka_inetd_task).
--author("anlicheng").
--include("efka_tables.hrl").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/4]).
--export([deploy/1]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE).
-
--record(state, {
-    service_root_dir :: string(),
-    task_id :: integer(),
-    service_id :: binary(),
-    tar_url :: binary()
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec deploy(Pid :: pid()) -> no_return().
-deploy(Pid) when is_pid(Pid) ->
-    gen_server:cast(Pid, deploy).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link(TaskId :: integer(), ServiceRootDir :: string(), ServiceId :: binary(), TarUrl :: binary()) ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link(TaskId, ServiceRootDir, ServiceId, TarUrl) when is_integer(TaskId), is_list(ServiceRootDir), is_binary(ServiceId), is_binary(TarUrl) ->
-    gen_server:start_link(?MODULE, [TaskId, ServiceRootDir, ServiceId, TarUrl], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([TaskId, ServiceRootDir, ServiceId, TarUrl]) ->
-    {ok, #state{task_id = TaskId, service_root_dir = ServiceRootDir, service_id = ServiceId, tar_url = TarUrl}}.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast(deploy, State = #state{task_id = TaskId, service_root_dir = ServiceRootDir, service_id = ServiceId, tar_url = TarUrl}) ->
-    do_deploy(TaskId, ServiceRootDir, ServiceId, TarUrl),
-    {stop, normal, State};
-handle_cast(_Request, State) ->
-    {stop, normal, State}.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_info(_Info, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(_Reason, _State = #state{}) ->
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec do_deploy(TaskId :: integer(), ServiceRootDir :: string(), ServiceId :: binary(), TarUrl :: binary()) -> no_return().
-do_deploy(TaskId, ServiceRootDir, ServiceId, TarUrl) when is_integer(TaskId), is_list(ServiceRootDir), is_binary(ServiceId), is_binary(TarUrl) ->
-    case download(binary_to_list(TarUrl), ServiceRootDir) of
-        {ok, TarFile, CostTs} ->
-            Log = io_lib:format("download: ~p completed, cost time: ~p(ms)", [binary_to_list(TarUrl), CostTs]),
-            efka_inetd_task_log:stash(TaskId, list_to_binary(Log)),
-
-            %% 创建工作目录
-            WorkDir = ServiceRootDir ++ "/work_dir/",
-            case filelib:ensure_dir(WorkDir) of
-                ok ->
-                    %% 清理目录下的文件
-                    catch delete_directory(WorkDir),
-                    case tar_extract(TarFile, WorkDir) of
-                        ok ->
-                            %% 更新数据
-                            ok = service_model:insert(#service{
-                                service_id = ServiceId,
-                                tar_url = TarUrl,
-                                %% 工作目录
-                                root_dir = ServiceRootDir,
-                                config_json = <<"">>,
-                                %% 状态： 0: 停止, 1: 运行中
-                                status = 0
-                            }),
-                            efka_inetd_task_log:stash(TaskId, <<"deploy success">>);
-                        {error, Reason} ->
-                            TarLog = io_lib:format("tar decompression: ~p, error: ~p", [filename:basename(TarFile), Reason]),
-                            efka_inetd_task_log:stash(TaskId, list_to_binary(TarLog))
-                    end;
-                {error, Reason} ->
-                    DownloadLog = io_lib:format("make work_dir error: ~p", [Reason]),
-                    efka_inetd_task_log:stash(TaskId, list_to_binary(DownloadLog))
-            end;
-        {error, Reason} ->
-            DownloadLog = io_lib:format("download: ~p, error: ~p", [binary_to_list(TarUrl), Reason]),
-            efka_inetd_task_log:stash(TaskId, list_to_binary(DownloadLog))
-    end.
-
-%% 递归删除目录下的问题
--spec delete_directory(string()) -> ok | {error, term()}.
-delete_directory(Dir) when is_list(Dir) ->
-    % 递归删除目录内容
-    case file:list_dir(Dir) of
-        {ok, Files} ->
-            lists:foreach(fun(File) ->
-                FullPath = filename:join(Dir, File),
-                case filelib:is_dir(FullPath) of
-                    true ->
-                        delete_directory(FullPath);
-                    false ->
-                        file:delete(FullPath)
-                end
-                          end, Files),
-            % 删除空目录
-            file:del_dir(Dir);
-        {error, enoent} ->
-            ok;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
-%% 解压文件到指定目录
--spec tar_extract(string(), string()) -> ok | {error, term()}.
-tar_extract(TarFile, TargetDir) when is_list(TarFile), is_list(TargetDir) ->
-    %% 判断文件的后缀名来判断, options: verbose
-    erl_tar:extract(TarFile, [compressed, {cwd, TargetDir}]).
-
-%% 下载文件
--spec download(Url :: string(), TargetDir :: string()) ->
-    {ok, TarFile :: string(), CostTs :: integer()} | {error, Reason :: term()}.
-download(Url, TargetDir) when is_list(Url), is_list(TargetDir) ->
-    SslOpts = [
-        {ssl, [
-            % 完全禁用证书验证
-            {verify, verify_none}
-        ]}
-    ],
-
-    TargetFile = get_filename_from_url(Url),
-    FullFilename = TargetDir ++ TargetFile,
-
-    StartTs = os:timestamp(),
-    case httpc:request(get, {Url, []}, SslOpts, [{sync, false}, {stream, self}]) of
-        {ok, RequestId} ->
-            case receive_data(RequestId, FullFilename) of
-                ok ->
-                    EndTs = os:timestamp(),
-                    %% 计算操作的时间，单位为毫秒
-                    CostMs = timer:now_diff(EndTs, StartTs) div 1000,
-                    {ok, FullFilename, CostMs};
-                {error, Reason} ->
-                    %% 出错需要删除掉文件
-                    file:delete(FullFilename),
-                    {error, Reason}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
-%% 处理头部信息, 解析可能的文件名
-receive_data(RequestId, FullFilename) ->
-    receive
-        {http, {RequestId, stream_start, _Headers}} ->
-            {ok, File} = file:open(FullFilename, [write, binary]),
-            receive_data0(RequestId, File);
-        {http, {RequestId, {{_, 404, Status}, _Headers, Body}}} ->
-            lager:debug("[efka_downloader] http_status: ~p, body: ~p", [Status, Body]),
-            {error, Status}
-    end.
-%% 接受文件数据
-receive_data0(RequestId, File) ->
-    receive
-        {http, {RequestId, {error, Reason}}} ->
-            ok = file:close(File),
-            {error, Reason};
-        {http, {RequestId, stream_end, _Headers}} ->
-            ok = file:close(File),
-            ok;
-        {http, {RequestId, stream, Data}} ->
-            file:write(File, Data),
-            receive_data0(RequestId, File)
-    end.
-
--spec get_filename_from_url(Url :: string()) -> string().
-get_filename_from_url(Url) when is_list(Url) ->
-    URIMap = uri_string:parse(Url),
-    Path = maps:get(path, URIMap),
-    filename:basename(Path).

apps/efka/src/efka_inetd_task_log.erl

@@ -1,136 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 09. 5月 2025 16:45
-%%%-------------------------------------------------------------------
--module(efka_inetd_task_log).
--author("anlicheng").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/0]).
--export([stash/2, flush/1, get_logs/1]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE).
-
--record(state, {
-    %% #{task_id => queue:new()}
-    pending_map = #{}
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec stash(TaskId :: integer(), Items :: binary() | [binary()]) -> no_return().
-stash(TaskId, Log) when is_integer(TaskId), is_binary(Log) ->
-    stash(TaskId, [Log]);
-stash(TaskId, Items) when is_integer(TaskId), is_list(Items) ->
-    {{Y, M, D}, {H, I, S}} = calendar:local_time(),
-    TimePrefix = iolist_to_binary(io_lib:format("[~b-~2..0b-~2..0b ~2..0b:~2..0b:~2..0b]", [Y, M, D, H, I, S])),
-    Log = iolist_to_binary([TimePrefix, <<" ">>, lists:join(<<" ">>, Items)]),
-    gen_server:cast(?SERVER, {stash, TaskId, Log}).
-
--spec flush(TaskId :: integer()) -> no_return().
-flush(TaskId) when is_integer(TaskId) ->
-    gen_server:cast(?SERVER, {flush, TaskId}).
-
--spec get_logs(TaskId :: integer()) -> {ok, Logs :: list()}.
-get_logs(TaskId) when is_integer(TaskId) ->
-    gen_server:call(?SERVER, {get_logs, TaskId}).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link() ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link() ->
-    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([]) ->
-    {ok, #state{}}.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_call({get_logs, TaskId}, _From, State = #state{pending_map = PendingMap}) ->
-    case maps:find(TaskId, PendingMap) of
-        error ->
-            Logs = task_log_model:get_logs(TaskId),
-            {reply, {ok, Logs}, State};
-        {ok, Q} ->
-            Logs = queue:to_list(Q),
-            {reply, {ok, Logs}, State}
-    end.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast({stash, TaskId, Log}, State = #state{pending_map = PendingMap}) ->
-    Q = maps:get(TaskId, PendingMap, queue:new()),
-    NQ = queue:in(Log, Q),
-    {noreply, State#state{pending_map = maps:put(TaskId, NQ, PendingMap)}};
-handle_cast({flush, TaskId}, State = #state{pending_map = PendingMap}) ->
-    case maps:take(TaskId, PendingMap) of
-        error ->
-            {noreply, State};
-        {Q, NPendingMap} ->
-            Logs = queue:to_list(Q),
-            ok = task_log_model:insert(TaskId, Logs),
-            {noreply, State#state{pending_map = NPendingMap}}
-    end.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_info(_Info, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(_Reason, _State = #state{}) ->
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================

apps/efka/src/efka_logger.erl

@@ -0,0 +1,179 @@
+%%%-------------------------------------------------------------------
+%%% @author aresei
+%%% @copyright (C) 2023, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 07. 9月 2023 17:07
+%%%-------------------------------------------------------------------
+-module(efka_logger).
+-author("aresei").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/1, write/1, write_lines/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+-record(state, {
+    file_name :: string(),
+    date :: calendar:date(),
+    file
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec write(Data :: binary()) -> ok.
+write(Data) when is_binary(Data) ->
+    gen_server:cast(?SERVER, {write, Data}).
+
+-spec write_lines(Lines :: [binary()]) -> ok.
+write_lines(Lines) when is_list(Lines) ->
+    gen_server:cast(?SERVER, {write_lines, Lines}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link(FileName :: string()) ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link(FileName) when is_list(FileName) ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [FileName], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([FileName]) ->
+    ensure_dir(),
+    FilePath = make_file(FileName),
+    {ok, File} = file:open(FilePath, [append, binary]),
+
+    {ok, #state{file = File, file_name = FileName, date = get_date()}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast({write, Data}, State = #state{file = OldFile, file_name = FileName, date = Date}) ->
+    Line = <<(time_prefix())/binary, " ", (format(Data))/binary, $\n>>,
+    case maybe_new_file(Date) of
+        true ->
+            file:close(OldFile),
+
+            FilePath = make_file(FileName),
+            {ok, File} = file:open(FilePath, [append, binary]),
+            ok = file:write(File, Line),
+            {noreply, State#state{file = File, date = get_date()}};
+        false ->
+            ok = file:write(OldFile, Line),
+            {noreply, State}
+    end;
+handle_cast({write_lines, Lines}, State = #state{file = OldFile, file_name = FileName, date = Date}) ->
+    Data = iolist_to_binary(lists:join(<<$\n>>, Lines)),
+    case maybe_new_file(Date) of
+        true ->
+            file:close(OldFile),
+
+            FilePath = make_file(FileName),
+            {ok, File} = file:open(FilePath, [append, binary]),
+            ok = file:write(File, Data),
+            {noreply, State#state{file = File, date = get_date()}};
+        false ->
+            ok = file:write(OldFile, Data),
+            {noreply, State}
+    end.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec format(binary() | [iodata()]) -> binary().
+format(Data) when is_binary(Data) ->
+    iolist_to_binary(Data);
+format(Items) when is_list(Items) ->
+    iolist_to_binary(lists:join(<<"\t">>, Items)).
+
+-spec time_prefix() -> binary().
+time_prefix() ->
+    {{Y, M, D}, {H, I, S}} = calendar:local_time(),
+    iolist_to_binary(io_lib:format("[~b-~2..0b-~2..0b ~2..0b:~2..0b:~2..0b]", [Y, M, D, H, I, S])).
+
+-spec make_file(LogFile :: string()) -> string().
+make_file(LogFile) when is_list(LogFile) ->
+    {Year, Month, Day} = erlang:date(),
+    Suffix = io_lib:format("~b~2..0b~2..0b", [Year, Month, Day]),
+    RootDir = code:root_dir() ++ "/log/",
+    lists:flatten(RootDir ++ LogFile ++ "." ++ Suffix).
+
+-spec ensure_dir() -> ok | {error, term()}.
+ensure_dir() ->
+    RootDir = code:root_dir() ++ "/log/",
+    case filelib:is_dir(RootDir) of
+        true ->
+            ok;
+        false ->
+            file:make_dir(RootDir)
+    end.
+
+%% 获取日期信息
+-spec get_date() -> Date :: calendar:date().
+get_date() ->
+    {Date, _} = calendar:local_time(),
+    Date.
+
+%% 通过日志判断是否需要生成新的日志文件
+-spec maybe_new_file(Date :: calendar:date()) -> boolean().
+maybe_new_file({Y, M, D}) ->
+    {{Y0, M0, D0}, _} = calendar:local_time(),
+    not (Y =:= Y0 andalso M =:= M0 andalso D =:= D0).

apps/efka/src/efka_manifest.erl

@@ -1,124 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%% 用于管理manifest.json配置文件
-%%% @end
-%%% Created : 05. 5月 2025 22:39
-%%%-------------------------------------------------------------------
--module(efka_manifest).
--author("anlicheng").
-
--record(manifest, {
-    work_dir = "" :: string(),
-    id = <<"">> :: binary(),
-    exec = <<"">>:: binary(),
-    args = [],
-    health_check = <<"">>
-}).
-
--type manifest() :: #manifest{}.
-
--export_type([manifest/0]).
-
-%% API
--export([new/1, startup/1]).
-
--spec new(ServiceRootDir :: string()) -> {ok, #manifest{}} | {error, Reason :: binary()}.
-new(ServiceRootDir) when is_list(ServiceRootDir) ->
-    WorkDir = ServiceRootDir ++ "/work_dir/",
-    case file:read_file(WorkDir ++ "manifest.json") of
-        {ok, ManifestInfo} ->
-            Settings = catch jiffy:decode(ManifestInfo, [return_maps]),
-            case check_manifest(Settings) of
-                {ok, Manifest} ->
-                    {ok, Manifest#manifest{work_dir = WorkDir}};
-                {error, Reason} ->
-                    {error, Reason}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec startup(Manifest :: #manifest{}) -> {ok, Port :: port()} | {error, Reason :: binary()}.
-startup(#manifest{id = Id, work_dir = WorkDir, exec = ExecCmd0, args = Args0}) ->
-    PortSettings = [
-        {cd, WorkDir},
-        {args, [binary_to_list(A) || A <- Args0]},
-        exit_status
-    ],
-    ExecCmd = binary_to_list(ExecCmd0),
-    RealExecCmd = filename:absname_join(WorkDir, ExecCmd),
-    lager:debug("[efka_manifest] service_id: ~p, real command is: ~p", [Id, RealExecCmd]),
-    case catch erlang:open_port({spawn_executable, RealExecCmd}, PortSettings) of
-        Port when is_port(Port) ->
-            {ok, Port};
-        _Other ->
-            {error, <<"exec command startup failed">>}
-    end.
-
-%% 检查配置是否合法
--spec check_manifest(Manifest :: map()) -> {ok, #manifest{}} | {error, Reason :: binary()}.
-check_manifest(Manifest) when is_map(Manifest) ->
-    RequiredKeys = [<<"id">>, <<"exec">>, <<"args">>, <<"health_check">>],
-    check_manifest0(RequiredKeys, Manifest, #manifest{});
-check_manifest(_Manifest) ->
-    {error, <<"invalid manifest json">>}.
-
-check_manifest0([], _Settings, Manifest) ->
-    {ok, Manifest};
-check_manifest0([<<"id">>|T], Settings, Manifest) ->
-    case maps:find(<<"id">>, Settings) of
-        error ->
-            {error, <<"miss service_id">>};
-        {ok, Id} when is_binary(Id) ->
-            check_manifest0(T, Settings, Manifest#manifest{id = Id});
-        {ok, _} ->
-            {error, <<"service_id is not string">>}
-    end;
-check_manifest0([<<"health_check">>|T], Settings, Manifest) ->
-    case maps:find(<<"health_check">>, Settings) of
-        error ->
-            {error, <<"miss health_check">>};
-        {ok, Url} when is_binary(Url) ->
-            case is_url(Url) of
-                true ->
-                    check_manifest0(T, Settings, Manifest#manifest{health_check = Url});
-                false ->
-                    {error, <<"health_check is not a invalid url">>}
-            end;
-        {ok, _} ->
-            {error, <<"health_check is not string">>}
-    end;
-check_manifest0([<<"exec">>|T], Settings, Manifest) ->
-    case maps:find(<<"exec">>, Settings) of
-        error ->
-            {error, <<"miss start">>};
-        {ok, Exec} when is_binary(Exec) ->
-            %% 不能包含空格
-            case binary:match(Exec, <<" ">>) of
-                nomatch ->
-                    check_manifest0(T, Settings, Manifest#manifest{exec = Exec});
-                _ ->
-                    {error, <<"start cmd cannot contain args">>}
-            end
-    end;
-check_manifest0([<<"args">>|T], Settings, Manifest) ->
-    case maps:find(<<"args">>, Settings) of
-        error ->
-            check_manifest0(T, Settings, Manifest#manifest{args = []});
-        %% 对参数项目不进行检查
-        {ok, Args} when is_list(Args) ->
-            check_manifest0(T, Settings, Manifest#manifest{args = Args});
-        {ok, _} ->
-            {error, <<"args must be list">>}
-    end.
-
--spec is_url(binary()) -> boolean().
-is_url(Input) when is_binary(Input) ->
-    try
-        uri_string:parse(Input),
-        true
-    catch
-        _:_ -> false
-    end.

apps/efka/src/efka_monitor.erl

@@ -1,25 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 03. 6月 2025 14:09
-%%%-------------------------------------------------------------------
--module(efka_monitor).
--author("anlicheng").
-
-%% API
--export([]).
-
-%% API
--export([memory_top/1, cpu_top/1, stop/0]).
-
-memory_top(Interval) when is_integer(Interval) ->
-    spawn(fun()->etop:start([{output, text}, {interval, Interval}, {lines, 20}, {sort, memory}])end).
-
-cpu_top(Interval) when is_integer(Interval) ->
-    spawn(fun()->etop:start([{output, text}, {interval, Interval}, {lines, 20}, {sort, runtime}])end).
-
-stop() ->
-    etop:stop().

apps/efka/src/efka_service.erl

@@ -1,291 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%% 1. 需要管理服务的整个生命周期，包括: 启动，停止
-%%% 2. 需要监控服务的状态，通过port的方式
-%%% 3. 服务的启动和关闭，需要在更高的层级控制
-%%% @end
-%%% Created : 18. 4月 2025 16:50
-%%%-------------------------------------------------------------------
--module(efka_service).
--author("anlicheng").
--include("efka_tables.hrl").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/2]).
--export([get_name/1, get_pid/1, attach_channel/2]).
--export([push_config/3, request_config/1, invoke/3]).
--export([metric_data/3, send_event/3]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--record(state, {
-    service_id :: binary(),
-    %% 通道id信息
-    channel_pid :: pid() | undefined,
-    %% 当前进程的port信息, OSPid = erlang:port_info(Port, os_pid)
-    port :: undefined | port(),
-    %% 系统对应的pid
-    os_pid :: undefined | integer(),
-    %% 配置信息
-    manifest :: undefined | efka_manifest:manifest(),
-    inflight = #{},
-
-    %% 映射关系: #{Ref => Fun}
-    callbacks = #{}
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec get_name(ServiceId :: binary()) -> atom().
-get_name(ServiceId) when is_binary(ServiceId) ->
-    list_to_atom("efka_service:" ++ binary_to_list(ServiceId)).
-
--spec get_pid(ServiceId :: binary()) -> undefined | pid().
-get_pid(ServiceId) when is_binary(ServiceId) ->
-    whereis(get_name(ServiceId)).
-
--spec push_config(Pid :: pid(), Ref :: reference(), ConfigJson :: binary()) -> no_return().
-push_config(Pid, Ref, ConfigJson) when is_pid(Pid), is_binary(ConfigJson) ->
-    gen_server:cast(Pid, {push_config, Ref, self(), ConfigJson}).
-
--spec invoke(Pid :: pid(), Ref :: reference(), Payload :: binary()) -> no_return().
-invoke(Pid, Ref, Payload) when is_pid(Pid), is_reference(Ref), is_binary(Payload) ->
-    gen_server:cast(Pid, {invoke, Ref, self(), Payload}).
-
--spec request_config(Pid :: pid()) -> {ok, Config :: binary()}.
-request_config(Pid) when is_pid(Pid) ->
-    gen_server:call(Pid, request_config).
-
--spec metric_data(Pid :: pid(), DeviceUUID :: binary(), Data :: binary()) -> no_return().
-metric_data(Pid, DeviceUUID, Data) when is_pid(Pid), is_binary(DeviceUUID), is_binary(Data) ->
-    gen_server:cast(Pid, {metric_data, DeviceUUID, Data}).
-
--spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> no_return().
-send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) ->
-    gen_server:cast(Pid, {send_event, EventType, Params}).
-
--spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}.
-attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) ->
-    gen_server:call(Pid, {attach_channel, ChannelPid}).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link(Name :: atom(), Service :: binary()) ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) ->
-    gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([ServiceId]) ->
-    %% supervisor进程通过exit(ChildPid, shutdown)调用的时候，确保terminate函数被调用
-    erlang:process_flag(trap_exit, true),
-    case service_model:get_service(ServiceId) of
-        error ->
-            lager:notice("[efka_service] service_id: ~p, not found", [ServiceId]),
-            ignore;
-        {ok, #service{root_dir = RootDir}} ->
-            %% 第一次启动，要求必须成功；只有第一次启动成功，后续的重启逻辑才有意义
-            case efka_manifest:new(RootDir) of
-                {ok, Manifest} ->
-                    case efka_manifest:startup(Manifest) of
-                        {ok, Port} ->
-                            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
-                            lager:debug("[efka_service] service: ~p, port: ~p, boot_service success os_pid: ~p", [ServiceId, Port, OSPid]),
-                            {ok, #state{service_id = ServiceId, manifest = Manifest, port = Port, os_pid = OSPid}};
-                        {error, Reason} ->
-                            lager:debug("[efka_service] service: ~p, boot_service get error: ~p", [ServiceId, Reason]),
-                            {stop, Reason}
-                    end;
-                {error, Reason} ->
-                    lager:notice("[efka_service] service: ~p, read manifest.json get error: ~p", [ServiceId, Reason]),
-                    ignore
-            end
-    end.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 绑定channel
-handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) ->
-    case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of
-        false ->
-            erlang:monitor(process, ChannelPid),
-            lager:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
-            {reply, ok, State#state{channel_pid = ChannelPid}};
-        true ->
-            {reply, {error, <<"channel exists">>}, State}
-    end;
-
-%% 请求参数项 done
-handle_call(request_config, _From, State = #state{service_id = ServiceId}) ->
-    case service_model:get_config_json(ServiceId) of
-        {ok, ConfigJson} ->
-            {reply, {ok, ConfigJson}, State};
-        error ->
-            {reply, {ok, <<>>}, State}
-    end;
-
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast({metric_data, DeviceUUID, LineProtocolData}, State = #state{service_id = ServiceId}) ->
-    lager:debug("[efka_service] metric_data service_id: ~p, device_uuid: ~p, metric data: ~p", [ServiceId, DeviceUUID, LineProtocolData]),
-    efka_agent:metric_data(ServiceId, DeviceUUID, LineProtocolData),
-    {noreply, State};
-
-handle_cast({send_event, EventType, Params}, State = #state{service_id = ServiceId}) ->
-    efka_agent:event(ServiceId, EventType, Params),
-    lager:debug("[efka_service] send_event, service_id: ~p, event_type: ~p, params: ~p", [ServiceId, EventType, Params]),
-    {noreply, State};
-
-%% 推送配置项目
-handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{channel_pid = ChannelPid, service_id = ServiceId, inflight = Inflight, callbacks = Callbacks}) ->
-    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
-        true ->
-            efka_tcp_channel:push_config(ChannelPid, Ref, self(), ConfigJson),
-            %% 设置成功，需要更新微服务的配置
-            CB = fun() -> service_model:set_config(ServiceId, ConfigJson) end,
-            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight), callbacks = maps:put(Ref, CB, Callbacks)}};
-        false ->
-            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
-            {noreply, State}
-    end;
-
-%% 推送配置项目
-handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{channel_pid = ChannelPid, inflight = Inflight}) ->
-    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
-        true ->
-            efka_tcp_channel:invoke(ChannelPid, Ref, self(), Payload),
-            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight)}};
-        false ->
-            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
-            {reply, State}
-    end;
-
-handle_cast(_Request, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 重启服务
-handle_info({timeout, _, reboot_service}, State = #state{service_id = ServiceId, manifest = Manifest}) ->
-    case efka_manifest:startup(Manifest) of
-        {ok, Port} ->
-            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
-            lager:debug("[efka_service] service_id: ~p, reboot success, port: ~p, os_pid: ~p", [ServiceId, Port, OSPid]),
-            {noreply, State#state{port = Port, os_pid = OSPid}};
-        {error, Reason} ->
-            lager:debug("[efka_service] service_id: ~p, boot_service get error: ~p", [ServiceId, Reason]),
-            try_reboot(),
-            {noreply, State}
-    end;
-
-%% 处理channel的回复
-handle_info({channel_reply, Ref, Reply}, State = #state{inflight = Inflight, callbacks = Callbacks}) ->
-    case maps:take(Ref, Inflight) of
-        error ->
-            {noreply, State};
-        {ReceiverPid, NInflight} ->
-            ReceiverPid ! {service_reply, Ref, Reply},
-
-            {noreply, State#state{inflight = NInflight, callbacks = trigger_callback(Ref, Callbacks)}}
-    end;
-
-handle_info({Port, {data, Data}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
-    lager:debug("[efka_service] service_id: ~p, port data: ~p", [ServiceId, Data]),
-    {noreply, State};
-
-%% 处理port的消息, Port的被动关闭会触发；因此这个时候的Port和State.port的值是相等的
-handle_info({Port, {exit_status, Code}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
-    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with code: ~p", [ServiceId, Port, Code]),
-    {noreply, State#state{port = undefined, os_pid = undefined}};
-
-%% 处理port的退出消息
-handle_info({'EXIT', Port, Reason}, State = #state{service_id = ServiceId}) when is_port(Port) ->
-    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with reason: ~p", [ServiceId, Port, Reason]),
-    try_reboot(),
-    {noreply, State#state{port = undefined, os_pid = undefined}};
-
-%% 处理channel进程的退出
-handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) ->
-    lager:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
-    {noreply, State#state{channel_pid = undefined, inflight = #{}}}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(Reason, _State = #state{service_id = ServiceId, port = Port, os_pid = OSPid}) ->
-    erlang:is_port(Port) andalso erlang:port_close(Port),
-    catch kill_os_pid(OSPid),
-    lager:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
-%% 关闭系统进程
--spec kill_os_pid(port() | undefined) -> no_return().
-kill_os_pid(undefined) ->
-    ok;
-kill_os_pid(OSPid) when is_integer(OSPid) ->
-    Cmd = lists:flatten(io_lib:format("kill -9 ~p", [OSPid])),
-    lager:debug("kill cmd is: ~p", [Cmd]),
-    os:cmd(Cmd).
-
--spec try_reboot() -> no_return().
-try_reboot() ->
-    erlang:start_timer(5000, self(), reboot_service).
-
--spec trigger_callback(Ref :: reference(), Callbacks :: map()) -> NewCallbacks :: map().
-trigger_callback(Ref, Callbacks) ->
-    case maps:take(Ref, Callbacks) of
-        error ->
-            Callbacks;
-        {Fun, NCallbacks} ->
-            catch Fun(),
-            NCallbacks
-    end.

apps/efka/src/efka_std_modbus_service.erl

@@ -1,291 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%% 1. 需要管理服务的整个生命周期，包括: 启动，停止
-%%% 2. 需要监控服务的状态，通过port的方式
-%%% 3. 服务的启动和关闭，需要在更高的层级控制
-%%% @end
-%%% Created : 18. 4月 2025 16:50
-%%%-------------------------------------------------------------------
--module(efka_std_modbus_service).
--author("anlicheng").
--include("efka_tables.hrl").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/2]).
--export([get_name/1, get_pid/1, attach_channel/2]).
--export([push_config/3, request_config/1, invoke/3]).
--export([metric_data/3, send_event/3]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--record(state, {
-    service_id :: binary(),
-    %% 通道id信息
-    channel_pid :: pid() | undefined,
-    %% 当前进程的port信息, OSPid = erlang:port_info(Port, os_pid)
-    port :: undefined | port(),
-    %% 系统对应的pid
-    os_pid :: undefined | integer(),
-    %% 配置信息
-    manifest :: undefined | efka_manifest:manifest(),
-    inflight = #{},
-
-    %% 映射关系: #{Ref => Fun}
-    callbacks = #{}
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec get_name(ServiceId :: binary()) -> atom().
-get_name(ServiceId) when is_binary(ServiceId) ->
-    list_to_atom("efka_service:" ++ binary_to_list(ServiceId)).
-
--spec get_pid(ServiceId :: binary()) -> undefined | pid().
-get_pid(ServiceId) when is_binary(ServiceId) ->
-    whereis(get_name(ServiceId)).
-
--spec push_config(Pid :: pid(), Ref :: reference(), ConfigJson :: binary()) -> no_return().
-push_config(Pid, Ref, ConfigJson) when is_pid(Pid), is_binary(ConfigJson) ->
-    gen_server:cast(Pid, {push_config, Ref, self(), ConfigJson}).
-
--spec invoke(Pid :: pid(), Ref :: reference(), Payload :: binary()) -> no_return().
-invoke(Pid, Ref, Payload) when is_pid(Pid), is_reference(Ref), is_binary(Payload) ->
-    gen_server:cast(Pid, {invoke, Ref, self(), Payload}).
-
--spec request_config(Pid :: pid()) -> {ok, Config :: binary()}.
-request_config(Pid) when is_pid(Pid) ->
-    gen_server:call(Pid, request_config).
-
--spec metric_data(Pid :: pid(), DeviceUUID :: binary(), Data :: binary()) -> no_return().
-metric_data(Pid, DeviceUUID, Data) when is_pid(Pid), is_binary(DeviceUUID), is_binary(Data) ->
-    gen_server:cast(Pid, {metric_data, DeviceUUID, Data}).
-
--spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> no_return().
-send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) ->
-    gen_server:cast(Pid, {send_event, EventType, Params}).
-
--spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}.
-attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) ->
-    gen_server:call(Pid, {attach_channel, ChannelPid}).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link(Name :: atom(), Service :: binary()) ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) ->
-    gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([ServiceId]) ->
-    %% supervisor进程通过exit(ChildPid, shutdown)调用的时候，确保terminate函数被调用
-    erlang:process_flag(trap_exit, true),
-    case service_model:get_service(ServiceId) of
-        error ->
-            lager:notice("[efka_service] service_id: ~p, not found", [ServiceId]),
-            ignore;
-        {ok, #service{root_dir = RootDir}} ->
-            %% 第一次启动，要求必须成功；只有第一次启动成功，后续的重启逻辑才有意义
-            case efka_manifest:new(RootDir) of
-                {ok, Manifest} ->
-                    case efka_manifest:startup(Manifest) of
-                        {ok, Port} ->
-                            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
-                            lager:debug("[efka_service] service: ~p, port: ~p, boot_service success os_pid: ~p", [ServiceId, Port, OSPid]),
-                            {ok, #state{service_id = ServiceId, manifest = Manifest, port = Port, os_pid = OSPid}};
-                        {error, Reason} ->
-                            lager:debug("[efka_service] service: ~p, boot_service get error: ~p", [ServiceId, Reason]),
-                            {stop, Reason}
-                    end;
-                {error, Reason} ->
-                    lager:notice("[efka_service] service: ~p, read manifest.json get error: ~p", [ServiceId, Reason]),
-                    ignore
-            end
-    end.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 绑定channel
-handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) ->
-    case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of
-        false ->
-            erlang:monitor(process, ChannelPid),
-            lager:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
-            {reply, ok, State#state{channel_pid = ChannelPid}};
-        true ->
-            {reply, {error, <<"channel exists">>}, State}
-    end;
-
-%% 请求参数项 done
-handle_call(request_config, _From, State = #state{service_id = ServiceId}) ->
-    case service_model:get_config_json(ServiceId) of
-        {ok, ConfigJson} ->
-            {reply, {ok, ConfigJson}, State};
-        error ->
-            {reply, {ok, <<>>}, State}
-    end;
-
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast({metric_data, DeviceUUID, LineProtocolData}, State = #state{service_id = ServiceId}) ->
-    lager:debug("[efka_service] metric_data service_id: ~p, device_uuid: ~p, metric data: ~p", [ServiceId, DeviceUUID, LineProtocolData]),
-    efka_agent:metric_data(ServiceId, DeviceUUID, LineProtocolData),
-    {noreply, State};
-
-handle_cast({send_event, EventType, Params}, State = #state{service_id = ServiceId}) ->
-    efka_agent:event(ServiceId, EventType, Params),
-    lager:debug("[efka_service] send_event, service_id: ~p, event_type: ~p, params: ~p", [ServiceId, EventType, Params]),
-    {noreply, State};
-
-%% 推送配置项目
-handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{channel_pid = ChannelPid, service_id = ServiceId, inflight = Inflight, callbacks = Callbacks}) ->
-    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
-        true ->
-            efka_tcp_channel:push_config(ChannelPid, Ref, self(), ConfigJson),
-            %% 设置成功，需要更新微服务的配置
-            CB = fun() -> service_model:set_config(ServiceId, ConfigJson) end,
-            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight), callbacks = maps:put(Ref, CB, Callbacks)}};
-        false ->
-            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
-            {noreply, State}
-    end;
-
-%% 推送配置项目
-handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{channel_pid = ChannelPid, inflight = Inflight}) ->
-    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
-        true ->
-            efka_tcp_channel:invoke(ChannelPid, Ref, self(), Payload),
-            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight)}};
-        false ->
-            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
-            {reply, State}
-    end;
-
-handle_cast(_Request, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 重启服务
-handle_info({timeout, _, reboot_service}, State = #state{service_id = ServiceId, manifest = Manifest}) ->
-    case efka_manifest:startup(Manifest) of
-        {ok, Port} ->
-            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
-            lager:debug("[efka_service] service_id: ~p, reboot success, port: ~p, os_pid: ~p", [ServiceId, Port, OSPid]),
-            {noreply, State#state{port = Port, os_pid = OSPid}};
-        {error, Reason} ->
-            lager:debug("[efka_service] service_id: ~p, boot_service get error: ~p", [ServiceId, Reason]),
-            try_reboot(),
-            {noreply, State}
-    end;
-
-%% 处理channel的回复
-handle_info({channel_reply, Ref, Reply}, State = #state{inflight = Inflight, callbacks = Callbacks}) ->
-    case maps:take(Ref, Inflight) of
-        error ->
-            {noreply, State};
-        {ReceiverPid, NInflight} ->
-            ReceiverPid ! {service_reply, Ref, Reply},
-
-            {noreply, State#state{inflight = NInflight, callbacks = trigger_callback(Ref, Callbacks)}}
-    end;
-
-handle_info({Port, {data, Data}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
-    lager:debug("[efka_service] service_id: ~p, port data: ~p", [ServiceId, Data]),
-    {noreply, State};
-
-%% 处理port的消息, Port的被动关闭会触发；因此这个时候的Port和State.port的值是相等的
-handle_info({Port, {exit_status, Code}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
-    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with code: ~p", [ServiceId, Port, Code]),
-    {noreply, State#state{port = undefined, os_pid = undefined}};
-
-%% 处理port的退出消息
-handle_info({'EXIT', Port, Reason}, State = #state{service_id = ServiceId}) when is_port(Port) ->
-    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with reason: ~p", [ServiceId, Port, Reason]),
-    try_reboot(),
-    {noreply, State#state{port = undefined, os_pid = undefined}};
-
-%% 处理channel进程的退出
-handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) ->
-    lager:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
-    {noreply, State#state{channel_pid = undefined, inflight = #{}}}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(Reason, _State = #state{service_id = ServiceId, port = Port, os_pid = OSPid}) ->
-    erlang:is_port(Port) andalso erlang:port_close(Port),
-    catch kill_os_pid(OSPid),
-    lager:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
-%% 关闭系统进程
--spec kill_os_pid(port() | undefined) -> no_return().
-kill_os_pid(undefined) ->
-    ok;
-kill_os_pid(OSPid) when is_integer(OSPid) ->
-    Cmd = lists:flatten(io_lib:format("kill -9 ~p", [OSPid])),
-    lager:debug("kill cmd is: ~p", [Cmd]),
-    os:cmd(Cmd).
-
--spec try_reboot() -> no_return().
-try_reboot() ->
-    erlang:start_timer(5000, self(), reboot_service).
-
--spec trigger_callback(Ref :: reference(), Callbacks :: map()) -> NewCallbacks :: map().
-trigger_callback(Ref, Callbacks) ->
-    case maps:take(Ref, Callbacks) of
-        error ->
-            Callbacks;
-        {Fun, NCallbacks} ->
-            catch Fun(),
-            NCallbacks
-    end.

apps/efka/src/efka_sup.erl

@@ -13,6 +13,7 @@
 
 -define(SERVER, ?MODULE).
 
+-spec start_link() -> {ok, pid()} | ignore | {error, term()}.
 start_link() ->
     supervisor:start_link({local, ?SERVER}, ?MODULE, []).
 
@@ -25,16 +26,35 @@ start_link() ->
 %%                  shutdown => shutdown(), % optional
 %%                  type => worker(),       % optional
 %%                  modules => modules()}   % optional
+-spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
 init([]) ->
     SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
     ChildSpecs = [
         #{
-            id => 'efka_inetd_task_log',
-            start => {'efka_inetd_task_log', start_link, []},
+            id => 'efka_logger',
+            start => {'efka_logger', start_link, ["deploy_log"]},
             restart => permanent,
             shutdown => 2000,
             type => worker,
-            modules => ['efka_inetd_task_log']
+            modules => ['efka_logger']
+        },
+
+        #{
+            id => 'efka_service_sup',
+            start => {'efka_service_sup', start_link, []},
+            restart => permanent,
+            shutdown => 2000,
+            type => supervisor,
+            modules => ['efka_service_sup']
+        },
+
+        #{
+            id => efka_service_model,
+            start => {efka_service_model, start_link, []},
+            restart => permanent,
+            shutdown => 5000,
+            type => worker,
+            modules => ['efka_service_model']
         },
 
         #{
@@ -47,52 +67,25 @@ init([]) ->
         },
 
         #{
-            id => 'efka_inetd',
-            start => {'efka_inetd', start_link, []},
+            id => 'efka_iot_client',
+            start => {'efka_iot_client', start_link, []},
             restart => permanent,
             shutdown => 2000,
             type => worker,
-            modules => ['efka_inetd']
+            modules => ['efka_iot_client']
         },
 
         #{
-            id => 'efka_agent',
-            start => {'efka_agent', start_link, []},
+            id => 'efka_iot_heartbeat',
+            start => {'efka_iot_heartbeat', start_link, []},
             restart => permanent,
             shutdown => 2000,
             type => worker,
-            modules => ['efka_agent']
-        },
-
-        #{
-            id => 'efka_tcp_sup',
-            start => {'efka_tcp_sup', start_link, []},
-            restart => permanent,
-            shutdown => 2000,
-            type => supervisor,
-            modules => ['efka_tcp_sup']
-        },
-
-        #{
-            id => 'efka_tcp_server',
-            start => {'efka_tcp_server', start_link, []},
-            restart => permanent,
-            shutdown => 2000,
-            type => worker,
-            modules => ['efka_tcp_server']
-        },
-
-        #{
-            id => 'efka_service_sup',
-            start => {'efka_service_sup', start_link, []},
-            restart => permanent,
-            shutdown => 2000,
-            type => supervisor,
-            modules => ['efka_service_sup']
+            modules => ['efka_iot_heartbeat']
         }
+
     ],
 
     {ok, {SupFlags, ChildSpecs}}.
 
 %% internal functions
-

apps/efka/src/efka_tcp_channel.erl

@@ -1,295 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 30. 4月 2025 09:22
-%%%-------------------------------------------------------------------
--module(efka_tcp_channel).
--author("anlicheng").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/1]).
--export([push_config/4, invoke/4]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE).
-
-%% 最大的等待时间
--define(PENDING_TIMEOUT, 10 * 1000).
-%% 消息类型
-
-%% 服务注册
--define(PACKET_REQUEST, 16#01).
-%% 消息响应
--define(PACKET_RESPONSE, 16#02).
-%% 上传数据
--define(PACKET_PUSH, 16#03).
-
--define(PACKET_PUB, 16#04).
-
--record(state, {
-    packet_id = 1,
-    socket :: gen_tcp:socket(),
-    service_id :: undefined | binary(),
-    service_pid :: undefined | pid(),
-    is_registered = false :: boolean(),
-
-    %% 请求响应的对应关系, #{packet_id => {ReceiverPid, Ref}}; 自身的inflight需要超时逻辑处理
-    inflight = #{}
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec push_config(ChannelPid :: pid(), Ref :: reference(), ReceiverPid :: pid(), ConfigJson :: binary()) -> no_return().
-push_config(ChannelPid, Ref, ReceiverPid, ConfigJson) when is_pid(ChannelPid), is_pid(ReceiverPid), is_binary(ConfigJson), is_reference(Ref) ->
-    gen_server:cast(ChannelPid, {push_config, Ref, ReceiverPid, ConfigJson}).
-
--spec invoke(ChannelPid :: pid(), Ref :: reference(), ReceiverPid :: pid(), Payload :: binary()) -> no_return().
-invoke(ChannelPid, Ref, ReceiverPid, Payload) when is_pid(ChannelPid), is_pid(ReceiverPid), is_binary(Payload), is_reference(Ref) ->
-    gen_server:cast(ChannelPid, {invoke, Ref, ReceiverPid, Payload}).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link(Socket :: gen_tcp:socket()) ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link(Socket) ->
-    gen_server:start_link(?MODULE, [Socket], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([Socket]) ->
-    ok = inet:setopts(Socket, [{active, true}]),
-    lager:debug("[efka_tcp_channel] get micro service socket: ~p", [Socket]),
-    {ok, #state{socket = Socket}}.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 推送配置项目
-handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) ->
-    PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"push_config">>, <<"params">> => #{<<"config">> => ConfigJson}},
-    Packet = jiffy:encode(PushConfig, [force_utf8]),
-    ok = gen_tcp:send(Socket, <<?PACKET_PUSH:8, Packet/binary>>),
-
-    erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}),
-    {noreply, State#state{packet_id = next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}};
-
-%% 远程调用
-handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) ->
-    PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"invoke">>, <<"params">> => #{<<"payload">> => Payload}},
-    Packet = jiffy:encode(PushConfig, [force_utf8]),
-    ok = gen_tcp:send(Socket, <<?PACKET_PUSH:8, Packet/binary>>),
-
-    erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}),
-    {noreply, State#state{packet_id = next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}};
-
-handle_cast(_Request, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 处理micro-client:request => efka 主动的请求
-handle_info({tcp, Socket, <<?PACKET_REQUEST:8, Data/binary>>}, State = #state{socket = Socket}) ->
-    Request = jiffy:decode(Data, [return_maps]),
-    case handle_request(Request, State) of
-        {ok, NewState} ->
-            {noreply, NewState};
-        {stop, Reason, NewState} ->
-            {stop, Reason, NewState}
-    end;
-
-%% 处理micro-client:response => efka 的响应
-handle_info({tcp, Socket, <<?PACKET_RESPONSE:8, Data/binary>>}, State = #state{socket = Socket, inflight = Inflight}) ->
-    Resp = jiffy:decode(Data, [return_maps]),
-    case Resp of
-        #{<<"id">> := Id, <<"result">> := Result} ->
-            case maps:take(Id, Inflight) of
-                error ->
-                    lager:warning("[tcp_channel] get unknown publish response message: ~p, packet_id: ~p", [Resp, Id]),
-                    {noreply, State};
-                {{ReceiverPid, Ref}, NInflight} ->
-                    case is_pid(ReceiverPid) andalso is_process_alive(ReceiverPid) of
-                        true ->
-                            ReceiverPid ! {channel_reply, Ref, {ok, Result}};
-                        false ->
-                            lager:warning("[tcp_channel] get publish response message: ~p, packet_id: ~p, but receiver_pid is deaded", [Resp, Id])
-                    end,
-                    {noreply, State#state{inflight = NInflight}}
-            end;
-        #{<<"id">> := Id, <<"error">> := #{<<"code">> := _Code, <<"message">> := Error}} ->
-            case maps:take(Id, Inflight) of
-                error ->
-                    lager:warning("[tcp_channel] get unknown publish response message: ~p, packet_id: ~p", [Resp, Id]),
-                    {noreply, State};
-                {{ReceiverPid, Ref}, NInflight} ->
-                    case is_pid(ReceiverPid) andalso is_process_alive(ReceiverPid) of
-                        true ->
-                            ReceiverPid ! {channel_reply, Ref, {error, Error}};
-                        false ->
-                            lager:warning("[tcp_channel] get publish response message: ~p, packet_id: ~p, but receiver_pid is deaded", [Resp, Id])
-                    end,
-                    {noreply, State#state{inflight = NInflight}}
-            end
-    end;
-
-%% 超时逻辑处理
-handle_info({timeout, _, {pending_timeout, Id}}, State = #state{inflight = Inflight}) ->
-    case maps:take(Id, Inflight) of
-        error ->
-            {noreply, State};
-        {{ReceiverPid, Ref}, NInflight} ->
-            case is_pid(ReceiverPid) andalso is_process_alive(ReceiverPid) of
-                true ->
-                    ReceiverPid ! {channel_reply, Ref, {error, <<"timeout">>}};
-                false ->
-                    ok
-            end,
-            {noreply, State#state{inflight = NInflight}}
-    end;
-
-%% 订阅的消息
-handle_info({topic_broadcast, Topic, Content}, State = #state{socket = Socket}) ->
-    Packet = jiffy:encode(#{<<"topic">> => Topic, <<"content">> => Content}, [force_utf8]),
-    ok = gen_tcp:send(Socket, <<?PACKET_PUB:8, Packet/binary>>),
-
-    {noreply, State};
-
-%% service进程关闭
-handle_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) ->
-    lager:debug("[tcp_channel] service_pid: ~p, exited: ~p", [ServicePid, Reason]),
-    {stop, normal, State#state{service_pid = undefined}};
-
-handle_info({tcp_error, Socket, Reason}, State = #state{socket = Socket, service_id = ServiceId}) ->
-    lager:debug("[tcp_channel] tcp_error: ~p, assoc service: ~p", [Reason, ServiceId]),
-    {stop, normal, State};
-handle_info({tcp_closed, Socket}, State = #state{socket = Socket, service_id = ServiceId}) ->
-    lager:debug("[tcp_channel] tcp_closed: ~p, assoc service: ~p", [Socket, ServiceId]),
-    {stop, normal, State}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(_Reason, _State = #state{}) ->
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
-%% 注册
-handle_request(#{<<"id">> := Id, <<"method">> := <<"register">>, <<"params">> := #{<<"service_id">> := ServiceId}}, State = #state{socket = Socket}) ->
-    case efka_service:get_pid(ServiceId) of
-        undefined ->
-            lager:warning("[efka_tcp_channel] service_id: ~p, not running", [ServiceId]),
-            Packet = json_error(Id, -1, <<"service not running">>),
-            ok = gen_tcp:send(Socket, <<?PACKET_RESPONSE:8, Packet/binary>>),
-            {stop, normal, State};
-        ServicePid when is_pid(ServicePid) ->
-            case efka_service:attach_channel(ServicePid, self()) of
-                ok ->
-                    Packet = json_result(Id, <<"ok">>),
-                    erlang:monitor(process, ServicePid),
-
-                    ok = gen_tcp:send(Socket, <<?PACKET_RESPONSE:8, Packet/binary>>),
-                    {ok, State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}};
-                {error, Error} ->
-                    lager:warning("[efka_tcp_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]),
-                    Packet = json_error(Id, -1, Error),
-                    ok = gen_tcp:send(Socket, <<?PACKET_RESPONSE:8, Packet/binary>>),
-                    {stop, normal, State}
-            end
-    end;
-
-%% 请求参数
-handle_request(#{<<"id">> := Id, <<"method">> := <<"request_config">>}, State = #state{socket = Socket, service_pid = ServicePid, is_registered = true}) ->
-    {ok, ConfigJson} = efka_service:request_config(ServicePid),
-    Packet = json_result(Id, ConfigJson),
-    ok = gen_tcp:send(Socket, <<?PACKET_RESPONSE:8, Packet/binary>>),
-    {ok, State};
-
-%% 数据项
-handle_request(#{<<"id">> := 0, <<"method">> := <<"metric_data">>, <<"params">> := #{<<"device_uuid">> := DeviceUUID, <<"metric">> := Metric}}, State = #state{service_pid = ServicePid, is_registered = true}) ->
-    efka_service:metric_data(ServicePid, DeviceUUID, Metric),
-    {ok, State};
-
-%% Event事件
-handle_request(#{<<"id">> := 0, <<"method">> := <<"event">>, <<"params">> := #{<<"event_type">> := EventType, <<"body">> := Body}}, State = #state{service_pid = ServicePid, is_registered = true}) ->
-    efka_service:send_event(ServicePid, EventType, Body),
-    {ok, State};
-
-%% 订阅事件
-handle_request(#{<<"id">> := 0, <<"method">> := <<"subscribe">>, <<"params">> := #{<<"topic">> := Topic}}, State = #state{is_registered = true}) ->
-    efka_subscription:subscribe(Topic, self()),
-    {ok, State}.
-
-%% 采用32位编码
--spec next_packet_id(PacketId :: integer()) -> NextPacketId :: integer().
-next_packet_id(PacketId) when PacketId >= 4294967295 ->
-    1;
-next_packet_id(PacketId) ->
-    PacketId + 1.
-
--spec json_result(Id :: integer(), Result :: term()) -> binary().
-json_result(Id, Result) when is_integer(Id) ->
-    Response = #{
-        <<"id">> => Id,
-        <<"result">> => Result
-    },
-    jiffy:encode(Response, [force_utf8]).
-
--spec json_error(Id :: integer(), Code :: integer(), Message :: binary()) -> binary().
-json_error(Id, Code, Message) when is_integer(Id), is_integer(Code), is_binary(Message) ->
-    Response = #{
-        <<"id">> => Id,
-        <<"error">> => #{
-            <<"code">> => Code,
-            <<"message">> => Message
-        }
-    },
-    jiffy:encode(Response, [force_utf8]).

apps/efka/src/efka_tcp_server.erl

@@ -1,46 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 29. 4月 2025 23:24
-%%%-------------------------------------------------------------------
--module(efka_tcp_server).
--author("anlicheng").
-
-%% API
--export([start_link/0, init/0]).
-
-start_link() ->
-    {ok, spawn_link(?MODULE, init, [])}.
-
-%% 监听循环
-init() ->
-    {ok, TcpServerProps} = application:get_env(efka, tcp_server),
-    Port = proplists:get_value(port, TcpServerProps),
-
-    case gen_tcp:listen(Port, [binary, {packet, 4}, {active, false}, {reuseaddr, true}]) of
-        {ok, ListenSocket} ->
-            lager:debug("[efka_tcp_server] Server started on port ~p~n", [Port]),
-            main_loop(ListenSocket);
-        {error, Reason} ->
-            lager:debug("[efka_tcp_server] Failed to start server: ~p~n", [Reason]),
-            exit(Reason)
-    end.
-
-main_loop(ListenSocket) ->
-    case gen_tcp:accept(ListenSocket) of
-        {ok, Socket} ->
-            % 为每个新连接生成一个处理进程
-            {ok, ChannelPid} = efka_tcp_sup:start_child(Socket),
-            ok = gen_tcp:controlling_process(Socket, ChannelPid),
-            % 继续监听下一个连接
-            main_loop(ListenSocket);
-        {error, closed} ->
-            lager:debug("[efka_tcp_server] Server socket closed"),
-            exit(tcp_closed);
-        {error, Reason} ->
-            lager:debug("[efka_tcp_server] Accept error: ~p", [Reason]),
-            exit(Reason)
-    end.

apps/efka/src/efka_tcp_sup.erl

@@ -1,43 +0,0 @@
-%%%-------------------------------------------------------------------
-%% @doc efka top level supervisor.
-%% @end
-%%%-------------------------------------------------------------------
-
--module(efka_tcp_sup).
-
--behaviour(supervisor).
-
--export([start_link/0, start_child/1]).
-
--export([init/1]).
-
--define(SERVER, ?MODULE).
-
-start_link() ->
-    supervisor:start_link({local, ?SERVER}, ?MODULE, []).
-
-%% sup_flags() = #{strategy => strategy(),         % optional
-%%                 intensity => non_neg_integer(), % optional
-%%                 period => pos_integer()}        % optional
-%% child_spec() = #{id => child_id(),       % mandatory
-%%                  start => mfargs(),      % mandatory
-%%                  restart => restart(),   % optional
-%%                  shutdown => shutdown(), % optional
-%%                  type => worker(),       % optional
-%%                  modules => modules()}   % optional
-init([]) ->
-    SupFlags = #{strategy => simple_one_for_one, intensity => 0, period => 1},
-    ChildSpec = #{
-        id => efka_tcp_channel,
-        start => {efka_tcp_channel, start_link, []},
-        restart => temporary,
-        type => worker
-    },
-    {ok, {SupFlags, [ChildSpec]}}.
-
-%% internal functions
-
-start_child(Socket) ->
-    supervisor:start_child(?MODULE, [Socket]).
-
-

apps/efka/src/efka_transport.erl

@@ -1,226 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 20. 4月 2025 18:47
-%%%-------------------------------------------------------------------
--module(efka_transport).
--author("anlicheng").
--include("message_pb.hrl").
--include("efka.hrl").
-
--behaviour(gen_server).
-
-%% API
--export([start_monitor/3]).
--export([connect/1, auth_request/2, send/3, async_call_reply/3, stop/1]).
--export([request/3]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE). 
-
--record(state, {
-    parent_pid :: pid(),
-    host :: string(),
-    port :: integer(),
-    socket :: undefined | ssl:sslsocket(),
-    packet_id = 1,
-    %% 通过packet_id建立请求和响应的关系
-    inflight = #{}
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec auth_request(Pid :: pid(), AuthBin :: binary()) -> no_return().
-auth_request(Pid, AuthBin) when is_pid(Pid), is_binary(AuthBin) ->
-    gen_server:cast(Pid, {auth_request, AuthBin}).
-
--spec request(Pid :: pid(), Method :: integer(), ReqBin :: binary()) -> Ref :: reference().
-request(Pid, Method, ReqBin) when is_pid(Pid), is_binary(ReqBin) ->
-    Ref = make_ref(),
-    gen_server:cast(Pid, {request, Ref, Method, ReqBin}),
-    Ref.
-
--spec connect(Pid :: pid()) -> no_return().
-connect(Pid) when is_pid(Pid) ->
-    gen_server:cast(Pid, connect).
-
--spec send(Pid :: pid(), Method :: integer(), Packet :: binary()) -> no_return().
-send(Pid, Method, Packet) when is_pid(Pid), is_integer(Method), is_binary(Packet) ->
-    gen_server:cast(Pid, {send, Method, Packet}).
-
--spec async_call_reply(Pid :: pid() | undefined, PacketId :: integer(), Response :: binary()) -> no_return().
-async_call_reply(undefined, PacketId, Response) when is_integer(PacketId), is_binary(Response) ->
-    ok;
-async_call_reply(Pid, PacketId, Response) when is_pid(Pid), is_integer(PacketId), is_binary(Response) ->
-    gen_server:cast(Pid, {async_call_reply, PacketId, Response}).
-
-%% 关闭的时候不一定能成功，可能关闭的时候；transport进程已经退出了
--spec stop(Pid :: pid() | undefined) -> ok.
-stop(undefined) ->
-    ok;
-stop(Pid) when is_pid(Pid) ->
-    catch gen_server:stop(Pid, normal, 2000).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_monitor(ParentPid :: pid(), Host :: string(), Port :: integer()) ->
-    {ok, {Pid :: pid(), MRef :: reference()}} | ignore | {error, Reason :: term()}).
-start_monitor(ParentPid, Host, Port) when is_pid(ParentPid), is_list(Host), is_integer(Port) ->
-    gen_server:start_monitor(?MODULE, [ParentPid, Host, Port], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([ParentPid, Host, Port]) ->
-    {ok, #state{parent_pid = ParentPid, host = Host, port = Port, socket = undefined}}.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_call(_Req, _From, State = #state{}) ->
-    {reply, ok, State#state{}}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 建立到目标服务器的连接
-handle_cast(connect, State = #state{host = Host, port = Port, parent_pid = ParentPid}) ->
-    SslOptions = [
-        binary,
-        {packet, 4},
-        {verify, verify_none}
-    ],
-    case ssl:connect(Host, Port, SslOptions, 5000) of
-        {ok, Socket} ->
-            ok = ssl:controlling_process(Socket, self()),
-            ParentPid ! {connect_reply, ok},
-            ping_ticker(),
-            {noreply, State#state{socket = Socket}};
-        {error, Reason} ->
-            ParentPid ! {connect_reply, {error, Reason}},
-            {noreply, State#state{socket = undefined}}
-    end;
-
-%% auth校验
-handle_cast({auth_request, AuthRequestBin}, State = #state{parent_pid = ParentPid, socket = Socket, packet_id = PacketId}) ->
-    ok = ssl:send(Socket, <<?PACKET_REQUEST, PacketId:32, ?METHOD_AUTH, AuthRequestBin/binary>>),
-    %% 需要等待auth返回的结果
-    receive
-        {ssl, Socket, <<?PACKET_RESPONSE, PacketId:32, ReplyBin/binary>>} ->
-            ParentPid ! {auth_reply, {ok, ReplyBin}},
-            {noreply, State#state{packet_id = PacketId + 1}};
-        {ssl, Socket, Info} ->
-            lager:warning("[efka_transport] get invalid auth_reply: ~p", [Info]),
-            ParentPid ! {auth_reply, {error, invalid_auth_reply}},
-            {noreply, State#state{packet_id = PacketId + 1}}
-    after 5000 ->
-        ParentPid ! {auth_reply, {error, timeout}},
-        {noreply, State#state{packet_id = PacketId + 1}}
-    end;
-
-%% 提交请求
-handle_cast({request, Ref, Method, ReqBin}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) ->
-    ok = ssl:send(Socket, <<?PACKET_REQUEST, PacketId:32, Method:8, ReqBin/binary>>),
-    {noreply, State#state{packet_id = PacketId + 1, inflight = maps:put(PacketId, Ref, Inflight)}};
-
-handle_cast({send, Method, Packet}, State = #state{socket = Socket}) ->
-    ok = ssl:send(Socket, <<?PACKET_REQUEST, Method:8, Packet/binary>>),
-    {noreply, State};
-
-%% 服务push的消息的回复
-handle_cast({async_call_reply, PacketId, Response}, State = #state{socket = Socket}) ->
-    ok = ssl:send(Socket, <<?PACKET_ASYNC_CALL_REPLY, PacketId:32, Response/binary>>),
-    {noreply, State}.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 服务器主动推送的数据，有packetId的是要求返回的；为0的表示不需要返回值
-handle_info({ssl, Socket, <<?PACKET_COMMAND, CommandType:8, Command/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
-    ParentPid ! {server_command, CommandType, Command},
-    {noreply, State};
-
-handle_info({ssl, Socket, <<?PACKET_PUB, PubBin/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
-    #pub{topic = Topic, content = Content} = message_pb:decode_msg(PubBin, pub),
-    ParentPid ! {server_pub, Topic, Content},
-    {noreply, State};
-
-handle_info({ssl, Socket, <<?PACKET_ASYNC_CALL, PacketId:32, AsyncCallBin/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
-    ParentPid ! {server_async_call, PacketId, AsyncCallBin},
-    {noreply, State};
-
-%% efka:request <-> iot:response
-handle_info({ssl, Socket, <<?PACKET_RESPONSE, PacketId:32, ReplyBin/binary>>}, State = #state{socket = Socket, inflight = Inflight, parent_pid = ParentPid}) ->
-    case maps:take(PacketId, Inflight) of
-        error ->
-            {noreply, State};
-        {Ref, NInflight} ->
-            ParentPid ! {server_reply, Ref, ReplyBin},
-            {noreply, State#state{inflight = NInflight}}
-    end;
-
-handle_info({ssl_error, Socket, Reason}, State = #state{socket = Socket}) ->
-    lager:debug("[efka_transport] ssl error: ~p", [Reason]),
-    {stop, normal, State};
-
-handle_info({ssl_closed, Socket}, State = #state{socket = Socket}) ->
-    {stop, normal, State};
-
-handle_info({timeout, _, ping_ticker}, State = #state{socket = Socket}) ->
-    ok = ssl:send(Socket, <<?PACKET_PING>>),
-    ping_ticker(),
-    {noreply, State};
-
-handle_info(Info, State = #state{}) ->
-    lager:notice("[efka_transport] get unknown info: ~p", [Info]),
-    {noreply, State}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-        State :: #state{}) -> term()).
-terminate(Reason, #state{}) ->
-    lager:notice("[efka_transport] terminate with reason: ~p", [Reason]),
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-        Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
-ping_ticker() ->
-    erlang:start_timer(5000, self(), ping_ticker).

apps/efka/src/efka_util.erl

@@ -14,26 +14,32 @@
 -export([timestamp/0, number_format/2, timestamp_ms/0, float_to_binary/2, int_format/2]).
 -export([chunks/2, rand_bytes/1, uuid/0, md5/1, sha_uuid/0]).
 -export([json_data/1, json_error/2]).
+-export([starts_with/2, file_md5/1]).
 
+-spec get_file_md5(string()) -> string().
 get_file_md5(FilePath) when is_list(FilePath) ->
     {ok, FileData} = file:read_file(FilePath),
     Md5Binary = crypto:hash(md5, FileData),
     string:lowercase(lists:flatten([io_lib:format("~2.16.0b", [X]) || X <- binary_to_list(Md5Binary)])).
 
 %% 时间，精确到毫秒
+-spec timestamp_ms() -> integer().
 timestamp_ms() ->
     {Mega, Seconds, Micro} = os:timestamp(),
     (Mega * 1000000 + Seconds) * 1000 + Micro div 1000.
 
+-spec timestamp() -> integer().
 timestamp() ->
     {Mega, Seconds, _Micro} = os:timestamp(),
     Mega * 1000000 + Seconds.
 
+-spec number_format(integer() | float(), integer()) -> integer() | float().
 number_format(Num, _Decimals) when is_integer(Num) ->
     Num;
 number_format(Float, Decimals) when is_float(Float) ->
     list_to_float(float_to_list(Float, [{decimals, Decimals}, compact])).
 
+-spec int_format(integer(), pos_integer()) -> integer().
 int_format(Num, Len) when is_integer(Num), Len > 0 ->
     S = integer_to_list(Num),
     case length(S) > Len of
@@ -49,6 +55,8 @@ chunks(List, Size) when is_list(List), is_integer(Size), Size > 0, length(List)
     [List];
 chunks(List, Size) when is_list(List), is_integer(Size), Size > 0 ->
     chunks0(List, Size, Size, [], []).
+
+-spec chunks0(list(), integer(), integer(), list(), [list()]) -> [list()].
 chunks0([], _, _, [], AccTarget) ->
     lists:reverse(AccTarget);
 chunks0([], _, _, Target, AccTarget) ->
@@ -58,19 +66,22 @@ chunks0(List, Size, 0, Target, AccTarget) ->
 chunks0([Hd | Tail], Size, Num, Target, AccTarget) ->
     chunks0(Tail, Size, Num - 1, [Hd | Target], AccTarget).
 
+-spec json_data(term()) -> binary().
 json_data(Data) ->
-    jiffy:encode(#{
+    iolist_to_binary(json:encode(#{
         <<"result">> => Data
-    }, [force_utf8]).
+    })).
 
+-spec json_error(integer(), binary()) -> binary().
 json_error(ErrCode, ErrMessage) when is_integer(ErrCode), is_binary(ErrMessage) ->
-    jiffy:encode(#{
+    iolist_to_binary(json:encode(#{
         <<"error">> => #{
             <<"code">> => ErrCode,
             <<"message">> => ErrMessage
         }
-    }, [force_utf8]).
+    })).
 
+-spec uuid() -> string().
 uuid() ->
     rand_bytes(16).
 
@@ -85,6 +96,7 @@ rand_bytes(Size) when is_integer(Size), Size > 0 ->
 md5(Str) when is_binary(Str) ->
     list_to_binary(lists:flatten([hex(X) || <<X:4>> <= erlang:md5(Str)])).
 
+-spec hex(0..15) -> byte().
 hex(N) when N < 10 ->
     $0 + N;
 hex(N) ->
@@ -102,4 +114,29 @@ float_to_binary(V, Decimals) when is_float(V), is_integer(Decimals) ->
 sha_uuid() ->
     Salt = crypto:strong_rand_bytes(32),
     Str = string:lowercase(binary:encode_hex(crypto:hash(sha256, Salt))),
-    binary:part(Str, 1, 32).
+    binary:part(Str, 1, 32).
+
+-spec starts_with(Binary :: binary(), Prefix :: binary()) -> boolean().
+starts_with(Binary, Prefix) when is_binary(Binary), is_binary(Prefix) ->
+    PrefixSize = byte_size(Prefix),
+    case Binary of
+        <<Prefix:PrefixSize/binary, _Rest/binary>> -> true;
+        _ -> false
+    end.
+
+-spec file_md5(FilePath :: string()) -> Md5 :: string().
+file_md5(FilePath) when is_list(FilePath) ->
+    {ok, F} = file:open(FilePath, [read, binary]),
+    Digest = md5_loop(F, crypto:hash_init(md5)),
+    file:close(F),
+    lists:flatten(io_lib:format("~32.16.0b", [binary:decode_unsigned(Digest)])).
+
+-spec md5_loop(file:io_device(), crypto:hash_state()) -> binary().
+md5_loop(F, Context) ->
+    %% 每次读取 1MB，可调整块大小
+    case file:read(F, 1024 * 1024) of
+        eof ->
+            crypto:hash_final(Context);
+        {ok, Bin} ->
+            md5_loop(F, crypto:hash_update(Context, Bin))
+    end.

apps/efka/src/iot/efka_iot_client.erl

@@ -0,0 +1,441 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 20. 4月 2026 00:00
+%%%-------------------------------------------------------------------
+-module(efka_iot_client).
+-author("anlicheng").
+-include("efka_tables.hrl").
+
+-behaviour(gen_statem).
+
+%% API
+-export([start_link/0]).
+-export([metric_data/2, ping/13, task_event_stream/3, close_task_event_stream/2]).
+-export([is_activated/0, dropped_message_count/0]).
+
+%% gen_statem callbacks
+-export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]).
+
+-define(SERVER, ?MODULE).
+
+%% 标记当前agent的状态，只有在 activated 状态下才可以正常的发送数据
+-define(STATE_DISCONNECTED, disconnected).
+%% 等待校验中
+-define(STATE_AUTH, auth).
+%% 激活状态下
+-define(STATE_ACTIVATED, activated).
+-define(SSL_PING_INTERVAL, 30000).
+-define(OUTBOX_SEGMENT_RECORD_LIMIT, 2000).
+-define(OUTBOX_MAX_SEGMENTS, 5).
+-define(OUTBOX_MAX_RECORD_BYTES, 16 * 1024 * 1024).
+
+-record(state, {
+    socket :: undefined | ssl:sslsocket(),
+    outbox :: efka_iot_outbox:outbox(),
+    %% 保存当前auth请求的ref，用来建立auth请求和响应的对应关系
+    auth_ref = undefined :: undefined | binary(),
+    ping_timer_ref = undefined :: undefined | reference(),
+    dropped_message_count = 0 :: non_neg_integer()
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+%% 发送数据
+-spec metric_data(RouteKey :: binary(), Metric :: binary()) -> ok.
+metric_data(RouteKey, Metric) when is_binary(RouteKey), is_binary(Metric) ->
+    gen_statem:cast(?SERVER, {metric_data, RouteKey, Metric}).
+
+-spec task_event_stream(TaskId :: integer(), Type :: binary(), Stream :: binary()) -> ok.
+task_event_stream(TaskId, Type, Stream) when is_integer(TaskId), is_binary(Type), is_binary(Stream) ->
+    gen_statem:cast(?SERVER, {task_event_stream, TaskId, Type, Stream}).
+
+-spec close_task_event_stream(TaskId :: integer(), Reason :: binary()) -> ok.
+close_task_event_stream(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) ->
+    gen_statem:cast(?SERVER, {close_task_event_stream, TaskId, Reason}).
+
+-spec is_activated() -> boolean().
+is_activated() ->
+    gen_statem:call(?SERVER, is_activated).
+
+-spec dropped_message_count() -> non_neg_integer().
+dropped_message_count() ->
+    gen_statem:call(?SERVER, dropped_message_count).
+
+-spec ping(term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term()) -> ok.
+ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) ->
+    gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}).
+
+-spec start_link() -> {ok, pid()} | ignore | {error, term()}.
+start_link() ->
+    gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_statem callbacks
+%%%===================================================================
+
+-spec init(list()) -> {ok, atom(), #state{}}.
+init([]) ->
+    case efka_iot_outbox:open(outbox_options()) of
+        {ok, Outbox} ->
+            erlang:start_timer(0, self(), create_transport),
+            {ok, ?STATE_DISCONNECTED, #state{socket = undefined, outbox = Outbox}};
+        {error, Reason} ->
+            {stop, Reason}
+    end.
+
+-spec callback_mode() -> handle_event_function.
+callback_mode() ->
+    handle_event_function.
+
+-spec outbox_options() -> map().
+outbox_options() ->
+    {ok, DetsDir} = application:get_env(efka, dets_dir),
+    #{
+        dir => filename:join(DetsDir, "iot_outbox"),
+        segment_record_limit => ?OUTBOX_SEGMENT_RECORD_LIMIT,
+        max_segments => ?OUTBOX_MAX_SEGMENTS,
+        max_record_bytes => ?OUTBOX_MAX_RECORD_BYTES
+    }.
+
+%% 异步发送数据，连接存在时直接发送；否则写入持久化 outbox。
+-spec handle_event(term(), term(), atom(), #state{}) -> term().
+handle_event(cast, {metric_data, RouteKey, Metric}, StateName, State = #state{socket = Socket}) ->
+    Packet = term_to_binary({<<"message">>, {<<"data">>, #{<<"route_key">> => RouteKey, <<"metric">> => Metric}}}),
+    case StateName of
+        ?STATE_ACTIVATED ->
+            ok = ssl:send(Socket, Packet),
+            {keep_state, State};
+        _ ->
+            case efka_iot_outbox:append(Packet, State#state.outbox) of
+                {ok, Outbox} ->
+                    {keep_state, State#state{outbox = Outbox}};
+                {dropped, capacity_reached, Outbox} ->
+                    logger:warning("[efka_iot_client] outbox capacity reached, drop offline metric"),
+                    {keep_state, State#state{
+                        outbox = Outbox,
+                        dropped_message_count = State#state.dropped_message_count + 1
+                    }};
+                {error, Reason} ->
+                    logger:warning("[efka_iot_client] append outbox failed, reason: ~p", [Reason]),
+                    {keep_state, State#state{dropped_message_count = State#state.dropped_message_count + 1}}
+            end
+    end;
+
+%% Task的stream流，只做实时的
+handle_event(cast, {task_event_stream, TaskId, Type, Stream}, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
+    logger:debug("[efka_iot_client] event_stream task_id: ~p, stream: ~ts", [TaskId, Stream]),
+    Packet = term_to_binary({<<"message">>, {<<"task_event">>, #{<<"task_id">> => TaskId, <<"type">> => Type, <<"stream">> => Stream}}}),
+    ok = ssl:send(Socket, Packet),
+    {keep_state, State};
+
+handle_event(cast, {close_task_event_stream, TaskId, Reason}, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
+    Packet = term_to_binary({<<"message">>, {<<"task_event">>, #{<<"task_id">> => TaskId, <<"type">> => <<"close">>, <<"stream">> => Reason}}}),
+    ok = ssl:send(Socket, Packet),
+    {keep_state, State};
+
+%% 其他情况下直接忽略
+handle_event(cast, _, _, State = #state{}) ->
+    {keep_state, State};
+
+handle_event({call, From}, is_activated, ?STATE_ACTIVATED, State = #state{}) ->
+    {keep_state, State, [{reply, From, true}]};
+handle_event({call, From}, is_activated, _StateName, State = #state{}) ->
+    {keep_state, State, [{reply, From, false}]};
+handle_event({call, From}, dropped_message_count, _StateName, State = #state{dropped_message_count = DroppedCount}) ->
+    {keep_state, State, [{reply, From, DroppedCount}]};
+
+%% 异步建立到服务器的连接
+handle_event(info, {timeout, _, create_transport}, ?STATE_DISCONNECTED, State) ->
+    case connect_socket() of
+        {ok, Socket} ->
+            Ref = request_ref(),
+            AuthPacket = auth_packet(Ref),
+            ok = ssl:send(Socket, AuthPacket),
+            logger:debug("[efka_iot_client] send auth request, ref: ~p", [Ref]),
+            {next_state, ?STATE_AUTH, State#state{socket = Socket, auth_ref = Ref}, [{state_timeout, 5000, auth_timeout}]};
+        {error, _Reason} ->
+            schedule_reconnect(),
+            {keep_state, State#state{socket = undefined}}
+    end;
+
+handle_event(state_timeout, auth_timeout, ?STATE_AUTH, State = #state{socket = Socket}) ->
+    logger:debug("[efka_iot_client] auth request timeout"),
+    disconnect(Socket),
+    schedule_reconnect(),
+    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined}};
+
+handle_event(info, {timeout, TimerRef, ssl_ping}, ?STATE_ACTIVATED, State = #state{socket = Socket, ping_timer_ref = TimerRef}) ->
+    Packet = term_to_binary({<<"message">>, <<"ping">>}),
+    case ssl:send(Socket, Packet) of
+        ok ->
+            {keep_state, schedule_ssl_ping(State)};
+        {error, Reason} ->
+            logger:warning("[efka_iot_client] send ssl ping failed, reason: ~p", [Reason]),
+            disconnect(Socket),
+            schedule_reconnect(),
+            {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}}
+    end;
+handle_event(info, {timeout, _TimerRef, ssl_ping}, _StateName, State) ->
+    {keep_state, State};
+
+%% 将缓存中的数据推送到服务器端
+handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
+    case efka_iot_outbox:next(State#state.outbox) of
+        {ok, Seq, Packet} ->
+            ok = ssl:send(Socket, Packet),
+            case efka_iot_outbox:ack(Seq, State#state.outbox) of
+                {ok, Outbox} ->
+                    {keep_state, State#state{outbox = Outbox}, [{next_event, info, flush_cache}]};
+                {error, Reason} ->
+                    logger:warning("[efka_iot_client] ack outbox failed, seq: ~p, reason: ~p", [Seq, Reason]),
+                    {keep_state, State}
+            end;
+        eof ->
+            {keep_state, State};
+        {error, Reason} ->
+            logger:warning("[efka_iot_client] read outbox failed, reason: ~p", [Reason]),
+            {keep_state, State}
+    end;
+handle_event(info, flush_cache, _, State) ->
+    {keep_state, State};
+
+%% 处理收到的ssl消息
+handle_event(info, {ssl, Socket, PacketBin}, _, State = #state{socket = Socket}) when is_binary(PacketBin) ->
+    try binary_to_term(PacketBin, [safe]) of
+        Packet ->
+            {keep_state, State, [{next_event, internal, Packet}]}
+    catch
+        error:Error ->
+            logger:warning("[efka_iot_client] binary_to_term get error: ~p, packet_size: ~p", [Error, byte_size(PacketBin)]),
+            disconnect(Socket),
+            cancel_ssl_ping(State),
+            schedule_reconnect(),
+            {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}}
+    end;
+handle_event(info, {ssl_error, Socket, Reason}, _, State = #state{}) ->
+    logger:debug("[efka_iot_client] ssl error: ~p", [Reason]),
+    disconnect(Socket),
+    cancel_ssl_ping(State),
+    schedule_reconnect(),
+    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}};
+handle_event(info, {ssl_closed, Socket}, _, State = #state{}) ->
+    logger:debug("[efka_iot_client] ssl closed"),
+    disconnect(Socket),
+    cancel_ssl_ping(State),
+    schedule_reconnect(),
+    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}};
+
+%%% 处理内部消息，ssl收到的消息会先 binary_to_term，再由这里按协议结构模式匹配
+
+%% 容器管理命令由 iot 发起，使用 command/command_response 语义。
+handle_event(internal, {<<"command">>, Ref, {<<"container">>, Request}}, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
+    handle_container_command(Ref, Request, Socket),
+    {keep_state, State};
+handle_event(internal, {<<"command">>, Ref, {<<"container">>, Request}}, _StateName, State = #state{socket = Socket}) ->
+    logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [Request]),
+    send_container_response(Socket, Ref, {error, <<"agent invalid">>}),
+    {keep_state, State};
+
+%% 处理response
+handle_event(internal, {<<"response">>, AuthRef, {<<"auth_response">>, <<"ok">>}}, ?STATE_AUTH, State = #state{auth_ref = AuthRef}) ->
+    logger:debug("[efka_iot_client] auth success"),
+    State1 = schedule_ssl_ping(State#state{auth_ref = undefined}),
+    {next_state, ?STATE_ACTIVATED, State1, [{next_event, info, flush_cache}]};
+handle_event(internal, {<<"response">>, AuthRef, {<<"auth_response">>, {<<"error">>, Reason}}}, ?STATE_AUTH, State = #state{socket = Socket, auth_ref = AuthRef}) ->
+    logger:debug("[efka_iot_client] auth failed, reason: ~p", [Reason]),
+    disconnect(Socket),
+    schedule_reconnect(),
+    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined}};
+handle_event(internal, {<<"response">>, _Ref, Reply}, StateName, State) ->
+    logger:warning("[efka_iot_client] ignore unexpected response in state ~p: ~p", [StateName, Reply]),
+    {keep_state, State};
+handle_event(internal, {<<"command_response">>, _Ref, Reply}, StateName, State) ->
+    logger:warning("[efka_iot_client] ignore unexpected command_response in state ~p: ~p", [StateName, Reply]),
+    {keep_state, State};
+
+%% 处理Pub/Sub机制
+handle_event(internal, {<<"message">>, <<"pong">>}, ?STATE_ACTIVATED, State) ->
+    {keep_state, State};
+handle_event(internal, {<<"message">>, {<<"pub">>, #{<<"topic">> := Topic, <<"qos">> := Qos, <<"content">> := Content}}}, ?STATE_ACTIVATED, State) ->
+    logger:debug("[efka_iot_client] get pub topic: ~p, qos: ~p, content: ~p", [Topic, Qos, Content]),
+    efka_subscription:publish(Topic, Qos, Content),
+    {keep_state, State};
+handle_event(internal, Packet, _StateName, State) ->
+    logger:warning("[efka_iot_client] ignore unknown packet: ~p", [Packet]),
+    {keep_state, State};
+
+handle_event(info, Info, _, State = #state{}) ->
+    logger:notice("[efka_iot_client] get unknown info: ~p", [Info]),
+    {keep_state, State}.
+
+-spec handle_container_command(binary(), term(), ssl:sslsocket()) -> ok.
+handle_container_command(Ref, #{<<"action">> := <<"list">>}, Socket) ->
+    Reply = docker_commands:get_containers(),
+    send_container_response(Socket, Ref, Reply),
+    ok;
+handle_container_command(Ref, #{<<"action">> := <<"deploy">>, <<"task_id">> := TaskId, <<"params">> := Params}, Socket) ->
+    Reply = docker_deploy_manager:deploy(TaskId, Params),
+    send_container_response(Socket, Ref, Reply),
+    ok;
+handle_container_command(Ref, #{<<"action">> := <<"start">>, <<"target">> := Target}, Socket) ->
+    Reply = docker_commands:start_container(container_target(Target)),
+    send_container_response(Socket, Ref, Reply),
+    ok;
+handle_container_command(Ref, #{<<"action">> := <<"stop">>, <<"target">> := Target, <<"timeout_seconds">> := TimeoutSeconds}, Socket) ->
+    Reply = docker_commands:stop_container(container_target(Target), TimeoutSeconds),
+    send_container_response(Socket, Ref, Reply),
+    ok;
+handle_container_command(Ref, #{<<"action">> := <<"kill">>, <<"target">> := Target, <<"signal">> := Signal}, Socket) ->
+    Reply = docker_commands:kill_container(container_target(Target), to_binary(Signal)),
+    send_container_response(Socket, Ref, Reply),
+    ok;
+handle_container_command(Ref, #{<<"action">> := <<"remove">>, <<"target">> := Target, <<"force">> := Force, <<"remove_volumes">> := RemoveVolumes}, Socket) ->
+    Reply = docker_commands:remove_container(container_target(Target), to_bool(Force), to_bool(RemoveVolumes)),
+    send_container_response(Socket, Ref, Reply),
+    ok;
+handle_container_command(Ref, #{<<"action">> := <<"config">>, <<"target">> := Target, <<"config">> := Config}, Socket) ->
+    Reply = docker_helper:update_container_config(container_target(Target), iolist_to_binary(Config)),
+    send_container_response(Socket, Ref, Reply),
+    ok;
+handle_container_command(Ref, Request, Socket) ->
+    logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [Request]),
+    send_container_response(Socket, Ref, {error, <<"agent invalid">>}),
+    ok.
+
+-spec terminate(term(), atom(), #state{}) -> ok.
+terminate(Reason, _StateName, State = #state{socket = Socket, outbox = Outbox}) ->
+    cancel_ssl_ping(State),
+    disconnect(Socket),
+    efka_iot_outbox:close(Outbox),
+    logger:notice("[efka_iot_client] terminate with reason: ~p", [Reason]),
+    ok.
+
+-spec code_change(term(), atom(), #state{}, term()) -> {ok, atom(), #state{}}.
+code_change(_OldVsn, StateName, State = #state{}, _Extra) ->
+    {ok, StateName, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec auth_packet(binary()) -> binary().
+auth_packet(Ref) when is_binary(Ref) ->
+    {ok, AuthInfo} = application:get_env(efka, auth),
+    UUID = proplists:get_value(uuid, AuthInfo),
+    Token = proplists:get_value(token, AuthInfo),
+
+    Timestamp = efka_util:timestamp(),
+    term_to_binary({<<"request">>, Ref, {<<"auth_request">>, #{
+        <<"uuid">> => list_to_binary(UUID),
+        <<"token">> => list_to_binary(Token),
+        <<"timestamp">> => Timestamp
+    }}}).
+
+-spec connect_socket() -> {ok, ssl:sslsocket()} | {error, term()}.
+connect_socket() ->
+    {ok, Props} = application:get_env(efka, iot_server),
+    Host = proplists:get_value(host, Props),
+    Port = proplists:get_value(tls_port, Props),
+    SslOptions = [
+        binary,
+        {active, true},
+        {packet, 4},
+        {verify, verify_none}
+    ],
+    ssl:connect(Host, Port, SslOptions, 5000).
+
+-spec disconnect(undefined | ssl:sslsocket()) -> ok.
+disconnect(undefined) ->
+    ok;
+disconnect(Socket) ->
+    catch ssl:close(Socket),
+    ok.
+
+-spec schedule_reconnect() -> reference().
+schedule_reconnect() ->
+    erlang:start_timer(5000, self(), create_transport).
+
+-spec schedule_ssl_ping(#state{}) -> #state{}.
+schedule_ssl_ping(State = #state{ping_timer_ref = TimerRef}) ->
+    cancel_timer(TimerRef),
+    State#state{ping_timer_ref = erlang:start_timer(?SSL_PING_INTERVAL, self(), ssl_ping)}.
+
+-spec cancel_ssl_ping(#state{}) -> ok.
+cancel_ssl_ping(#state{ping_timer_ref = TimerRef}) ->
+    cancel_timer(TimerRef).
+
+-spec cancel_timer(undefined | reference()) -> ok.
+cancel_timer(undefined) ->
+    ok;
+cancel_timer(TimerRef) ->
+    _ = erlang:cancel_timer(TimerRef),
+    ok.
+
+-spec request_ref() -> binary().
+request_ref() ->
+    crypto:strong_rand_bytes(16).
+
+-spec send_container_response(ssl:sslsocket(), binary(), term()) -> ok.
+send_container_response(Socket, Ref, Reply) ->
+    Packet = term_to_binary({<<"command_response">>, Ref, {<<"container">>, safe_reply(Reply)}}),
+    ok = ssl:send(Socket, Packet).
+
+-spec safe_reply(term()) -> term().
+safe_reply(ok) ->
+    <<"ok">>;
+safe_reply({ok, Result}) ->
+    {<<"ok">>, safe_term(Result)};
+safe_reply({error, Reason}) ->
+    {<<"error">>, safe_term(Reason)}.
+
+-spec safe_term(term()) -> term().
+safe_term(true) ->
+    true;
+safe_term(false) ->
+    false;
+safe_term(undefined) ->
+    undefined;
+safe_term(Value) when is_atom(Value) ->
+    atom_to_binary(Value, utf8);
+safe_term(Value) when is_map(Value) ->
+    maps:from_list([{safe_term(K), safe_term(V)} || {K, V} <- maps:to_list(Value)]);
+safe_term(Value) when is_list(Value) ->
+    [safe_term(Item) || Item <- Value];
+safe_term(Value) when is_tuple(Value) ->
+    list_to_tuple([safe_term(Item) || Item <- tuple_to_list(Value)]);
+safe_term(Value) ->
+    Value.
+
+-spec container_target(map()) -> binary().
+container_target(Target) when is_map(Target) ->
+    NameBin = to_binary(maps:get(<<"name">>, Target, <<>>)),
+    IdBin = to_binary(maps:get(<<"id">>, Target, <<>>)),
+    case NameBin of
+        <<>> ->
+            true = IdBin =/= <<>>,
+            IdBin;
+        _ ->
+            NameBin
+    end.
+
+-spec to_binary(binary() | list()) -> binary().
+to_binary(Value) when is_binary(Value) ->
+    Value;
+to_binary(Value) when is_list(Value) ->
+    unicode:characters_to_binary(Value).
+
+-spec to_bool(true | false | 0 | 1) -> boolean().
+to_bool(true) ->
+    true;
+to_bool(1) ->
+    true;
+to_bool(false) ->
+    false;
+to_bool(0) ->
+    false.

apps/efka/src/iot/efka_iot_heartbeat.erl

@@ -0,0 +1,119 @@
+%%%-------------------------------------------------------------------
+%%% @doc UDP heartbeat sender for iot host liveness.
+%%% @end
+%%%-------------------------------------------------------------------
+-module(efka_iot_heartbeat).
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+-define(HEARTBEAT_VERSION, 1).
+-define(HEARTBEAT_NONCE_BYTES, 16).
+-define(DEFAULT_INTERVAL, 5000).
+
+-record(state, {
+    socket :: gen_udp:socket(),
+    host :: inet:hostname() | inet:ip_address(),
+    port :: inet:port_number(),
+    interval :: pos_integer(),
+    uuid :: binary(),
+    heartbeat_secret :: binary()
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec start_link() -> {ok, pid()} | ignore | {error, term()}.
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+-spec init(list()) -> {ok, #state{}} | {stop, term()}.
+init([]) ->
+    ok = application:ensure_started(crypto),
+    {ok, ServerProps} = application:get_env(efka, iot_server),
+    {ok, AuthProps} = application:get_env(efka, auth),
+    HeartbeatProps = case application:get_env(efka, heartbeat) of
+        {ok, Props} -> Props;
+        undefined -> []
+    end,
+    Host = proplists:get_value(host, ServerProps),
+    UdpPort = proplists:get_value(udp_port, ServerProps),
+    Interval = proplists:get_value(interval, HeartbeatProps, ?DEFAULT_INTERVAL),
+    UUID = list_to_binary(proplists:get_value(uuid, AuthProps)),
+    Token = list_to_binary(proplists:get_value(token, AuthProps)),
+    HeartbeatSecret = crypto:hash(sha256, Token),
+    case gen_udp:open(0, [binary]) of
+        {ok, Socket} ->
+            erlang:send_after(0, self(), heartbeat),
+            {ok, #state{
+                socket = Socket,
+                host = Host,
+                port = UdpPort,
+                interval = Interval,
+                uuid = UUID,
+                heartbeat_secret = HeartbeatSecret
+            }};
+        {error, Reason} ->
+            {stop, Reason}
+    end.
+
+-spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, ok, #state{}}.
+handle_call(_Request, _From, State) ->
+    {reply, ok, State}.
+
+-spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
+handle_cast(_Request, State) ->
+    {noreply, State}.
+
+-spec handle_info(term(), #state{}) -> {noreply, #state{}}.
+handle_info(heartbeat, State = #state{interval = Interval}) ->
+    ok = send_heartbeat(State),
+    erlang:send_after(Interval, self(), heartbeat),
+    {noreply, State};
+handle_info(Info, State) ->
+    logger:warning("[efka_iot_heartbeat] ignore unknown info: ~p", [Info]),
+    {noreply, State}.
+
+-spec terminate(term(), #state{}) -> ok.
+terminate(_Reason, #state{socket = Socket}) ->
+    gen_udp:close(Socket),
+    ok.
+
+-spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
+code_change(_OldVsn, State, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec send_heartbeat(#state{}) -> ok.
+send_heartbeat(#state{socket = Socket, host = Host, port = Port, uuid = UUID, heartbeat_secret = HeartbeatSecret}) ->
+    Packet = heartbeat_packet_iodata(UUID, HeartbeatSecret),
+    case gen_udp:send(Socket, Host, Port, Packet) of
+        ok ->
+            ok;
+        {error, Reason} ->
+            logger:warning("[efka_iot_heartbeat] send heartbeat failed, reason: ~p", [Reason]),
+            ok
+    end.
+
+-spec heartbeat_packet_iodata(binary(), binary()) -> iodata().
+heartbeat_packet_iodata(UUID, HeartbeatSecret) when is_binary(UUID), is_binary(HeartbeatSecret) ->
+    UUIDLen = byte_size(UUID),
+    Timestamp = efka_util:timestamp(),
+    Nonce = crypto:strong_rand_bytes(?HEARTBEAT_NONCE_BYTES),
+    Payload = <<?HEARTBEAT_VERSION:8, UUIDLen:16, UUID:UUIDLen/binary, Timestamp:64/unsigned-big, Nonce/binary>>,
+    Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload),
+    [Payload, Mac].

apps/efka/src/iot/efka_iot_outbox.erl

@@ -0,0 +1,662 @@
+%%%-------------------------------------------------------------------
+%%% @doc
+%%% efka_iot_client 的持久化待发送队列。
+%%%
+%%% 本模块用于保存 efka_iot_client 离线期间产生的待发送 packet。底层使用
+%%% 追加写 segment 文件的方式实现。当前场景只有一个写入方和一个读取方，并且
+%%% 读取方每次只读取一条数据。
+%%%
+%%% 文件结构：
+%%% - `segment-N.log'：一个 segment 文件，按写入顺序追加保存 record。
+%%% - `metadata.json'：使用原始 JSON 文本保存 write_seq、acked_seq 和
+%%%   writer_segment。进程启动时仍然会扫描 segment 文件，并以 segment 文件
+%%%   里的真实数据作为准确信息；因此即使 metadata 落后，也不会导致跳过或删除
+%%%   未消费的数据。
+%%% - outbox 目录通过 open/1 的 dir 参数传入，本模块不直接读取应用环境变量。
+%%%
+%%% open/1 参数：
+%%% - dir：outbox 数据目录。
+%%% - segment_record_limit：每个 segment 最多保存的 record 数。
+%%% - max_segments：最多保留的 live segment 数。
+%%% - max_record_bytes：允许读取的单条 record 最大 payload 字节数。
+%%%
+%%% record 格式：
+%%% - 8 字节 magic bytes，当前为 <<"EFKAOBX1">>。
+%%% - 1 字节 version，当前为 1。
+%%% - 1 字节 header size，当前为 22。
+%%% - 4 字节 unsigned big-endian packet size。
+%%% - 8 字节 unsigned big-endian Seq。
+%%% - Packet 原始二进制内容。
+%%% - 4 字节 unsigned big-endian crc32，校验范围为 header + Packet。
+%%% - 读取时会先搜索 magic，再校验 version、header size、packet size 和
+%%%   crc32。遇到损坏 record 时，会继续向后寻找下一个合法 magic，尽量恢复
+%%%   后续可读数据。
+%%%
+%%% 写入逻辑：
+%%% - `append/2' 将一条 record 写入当前 writer segment，随后 fsync 当前
+%%%   segment，推进 write_seq，并持久化 metadata。
+%%% - 每个 segment 最多保存多少条数据由 open/1 的 segment_record_limit
+%%%   参数决定。
+%%% - outbox 最多保留多少个 live segment 由 open/1 的 max_segments 参数
+%%%   决定。当当前 segment 已满，且 live segment 数已经达到上限时，新
+%%%   packet 会被丢弃，已有磁盘数据保持不变。
+%%% - 当当前 segment 已满，但 live segment 数量尚未达到上限时，writer 会
+%%%   关闭旧文件，并开始写入 `segment-(N + 1).log'。
+%%%
+%%% 读取逻辑：
+%%% - `next/1' 找到第一个 end_seq 大于 acked_seq 的 live segment，打开该
+%%%   segment，并只返回第一条 Seq 大于 acked_seq 的 record。
+%%% - 本模块不做批量读取；efka_iot_client 每次只发送并 ack 一条 packet。
+%%% - `ack/2' 接受当前读取到的 Seq。正常情况下 Seq 连续递增；如果某条
+%%%   record 损坏并被读取端跳过，ack 允许推进到后续合法 record 的 Seq，
+%%%   避免单条坏数据阻断整个 segment 后续数据的消费。
+%%%
+%%% segment 清理逻辑：
+%%% - 当 `ack/2' 消费完某个 segment 的最后一条 record，并且后续 segment
+%%%   里还有未读数据时，已消费的 segment 文件会被删除；下一次读取会从后续
+%%%   segment 开始。
+%%% - 当 acked_seq 追上 write_seq 时，说明所有 record 都已经投递完成；此时
+%%%   会删除所有 segment 文件，将计数器重置为 0，下一次 append 从全新的
+%%%   `segment-1.log' 的 Seq 1 开始。
+%%% @end
+%%%-------------------------------------------------------------------
+-module(efka_iot_outbox).
+
+-export([open/1, close/1, append/2, next/1, ack/2]).
+-export_type([outbox/0]).
+
+-record(segment, {
+    id :: pos_integer(),
+    path :: file:filename_all(),
+    start_seq = 0 :: non_neg_integer(),
+    end_seq = 0 :: non_neg_integer(),
+    records = 0 :: non_neg_integer()
+}).
+
+-record(outbox, {
+    dir :: file:filename_all(),
+    metadata_path :: file:filename_all(),
+    writer_segment = 1 :: pos_integer(),
+    fd :: file:fd(),
+    segment_record_limit :: pos_integer(),
+    max_segments :: pos_integer(),
+    max_record_bytes :: pos_integer(),
+    segments = [] :: [#segment{}],
+    next_seq = 1 :: pos_integer(),
+    write_seq = 0 :: non_neg_integer(),
+    acked_seq = 0 :: non_neg_integer()
+}).
+
+-type outbox() :: #outbox{}.
+
+-define(METADATA_FILE, "metadata.json").
+-define(SEGMENT_PREFIX, "segment-").
+-define(SEGMENT_EXT, ".log").
+-define(RECORD_MAGIC, <<"EFKAOBX1">>).
+-define(RECORD_MAGIC_SIZE, 8).
+-define(RECORD_VERSION, 1).
+-define(RECORD_HEADER_SIZE, 22).
+-define(RECORD_CRC_SIZE, 4).
+
+-type open_options() :: #{
+    dir := file:filename_all(),
+    segment_record_limit := pos_integer(),
+    max_segments := pos_integer(),
+    max_record_bytes := pos_integer()
+}.
+
+-spec open(open_options()) -> {ok, outbox()} | {error, term()}.
+open(Options) when is_map(Options) ->
+    Dir = maps:get(dir, Options),
+    SegmentRecordLimit = positive_option(segment_record_limit, Options),
+    MaxSegments = positive_option(max_segments, Options),
+    MaxRecordBytes = positive_option(max_record_bytes, Options),
+    MetadataPath = filename:join(Dir, ?METADATA_FILE),
+    maybe
+        ok ?= ensure_dir(Dir),
+        {_MetaWriteSeq, MetaAckedSeq, MetaWriterSegment} = read_metadata(MetadataPath),
+        {ok, Segments} ?= load_segments(Dir, MaxRecordBytes),
+        ScannedWriteSeq = max_segment_end(Segments),
+        WriteSeq = ScannedWriteSeq,
+        AckedSeq = min(MetaAckedSeq, WriteSeq),
+        WriterSegment = writer_segment_id(Segments, MetaWriterSegment),
+        {ok, Fd} ?= open_writer(segment_path(Dir, WriterSegment)),
+        Outbox0 = #outbox{
+            dir = Dir,
+            metadata_path = MetadataPath,
+            writer_segment = WriterSegment,
+            fd = Fd,
+            segment_record_limit = SegmentRecordLimit,
+            max_segments = MaxSegments,
+            max_record_bytes = MaxRecordBytes,
+            segments = Segments,
+            next_seq = WriteSeq + 1,
+            write_seq = WriteSeq,
+            acked_seq = AckedSeq
+        },
+        {ok, Outbox1} ?= normalize_open_outbox(Outbox0),
+        ok ?= persist_metadata(Outbox1),
+        {ok, Outbox1}
+    else
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec close(outbox()) -> ok.
+close(#outbox{fd = Fd}) ->
+    _ = file:close(Fd),
+    ok.
+
+-spec append(binary(), outbox()) ->
+    {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}.
+append(Packet, Outbox0) when is_binary(Packet) ->
+    case ensure_writable_segment(Outbox0) of
+        {ok, Outbox1 = #outbox{fd = Fd, next_seq = Seq}} ->
+            Record = encode_record(Seq, Packet),
+            maybe
+                ok ?= file:write(Fd, Record),
+                ok ?= file:sync(Fd),
+                Outbox2 = mark_written(Outbox1, Seq),
+                ok ?= persist_metadata(Outbox2),
+                {ok, Outbox2}
+            else
+                {error, Reason} ->
+                    {error, Reason}
+            end;
+        {dropped, capacity_reached, Outbox1} ->
+            {dropped, capacity_reached, Outbox1};
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec next(outbox()) -> eof | {ok, pos_integer(), binary()} | {error, term()}.
+next(#outbox{segments = Segments, acked_seq = AckedSeq, max_record_bytes = MaxRecordBytes}) ->
+    case next_segment(Segments, AckedSeq) of
+        undefined ->
+            eof;
+        #segment{path = Path} ->
+            case file:open(Path, [read, binary]) of
+                {ok, Fd} ->
+                    Result = read_next_unacked(Fd, AckedSeq, MaxRecordBytes),
+                    _ = file:close(Fd),
+                    Result;
+                {error, enoent} ->
+                    eof;
+                {error, Reason} ->
+                    {error, Reason}
+            end
+    end.
+
+-spec ack(pos_integer(), outbox()) -> {ok, outbox()} | {error, term()}.
+ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq =< AckedSeq ->
+    {ok, Outbox};
+ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq > AckedSeq ->
+    AckedOutbox = Outbox#outbox{acked_seq = Seq},
+    maybe
+        ok ?= persist_metadata(AckedOutbox),
+        {ok, NOutbox} ?= maybe_reset_or_prune(AckedOutbox),
+        ok ?= persist_metadata(NOutbox),
+        {ok, NOutbox}
+    else
+        {error, Reason} ->
+            {error, Reason}
+    end;
+ack(Seq, #outbox{acked_seq = AckedSeq}) when is_integer(Seq) ->
+    {error, {invalid_ack, Seq, AckedSeq}}.
+
+-spec ensure_dir(file:filename_all()) -> ok | {error, term()}.
+ensure_dir(Dir) ->
+    filelib:ensure_dir(filename:join(Dir, "dummy")).
+
+-spec positive_option(atom(), map()) -> pos_integer().
+positive_option(Key, Options) ->
+    case maps:get(Key, Options) of
+        Value when is_integer(Value), Value > 0 ->
+            Value;
+        Value ->
+            error({invalid_option, Key, Value})
+    end.
+
+-spec read_metadata(file:filename_all()) ->
+    {non_neg_integer(), non_neg_integer(), pos_integer()}.
+read_metadata(MetadataPath) ->
+    case file:read_file(MetadataPath) of
+        {ok, Bin} ->
+            decode_metadata(Bin);
+        {error, _} ->
+            {0, 0, 1}
+    end.
+
+-spec decode_metadata(binary()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}.
+decode_metadata(Bin) ->
+    try json:decode(Bin) of
+        Metadata ->
+            safe_metadata(Metadata)
+    catch
+        _:_Reason ->
+            {0, 0, 1}
+    end.
+
+-spec safe_metadata(term()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}.
+safe_metadata(#{
+    <<"write_seq">> := WriteSeq,
+    <<"acked_seq">> := AckedSeq,
+    <<"writer_segment">> := WriterSegment
+})
+  when is_integer(WriteSeq), WriteSeq >= 0,
+       is_integer(AckedSeq), AckedSeq >= 0,
+       is_integer(WriterSegment), WriterSegment > 0 ->
+    {WriteSeq, AckedSeq, WriterSegment};
+safe_metadata(#{<<"write_seq">> := WriteSeq, <<"acked_seq">> := AckedSeq})
+  when is_integer(WriteSeq), WriteSeq >= 0, is_integer(AckedSeq), AckedSeq >= 0 ->
+    {WriteSeq, AckedSeq, 1};
+safe_metadata(_) ->
+    {0, 0, 1}.
+
+-spec persist_metadata(outbox()) -> ok | {error, term()}.
+persist_metadata(#outbox{
+    metadata_path = MetadataPath,
+    writer_segment = WriterSegment,
+    write_seq = WriteSeq,
+    acked_seq = AckedSeq
+}) ->
+    Metadata = iolist_to_binary(json:encode(#{
+        <<"write_seq">> => WriteSeq,
+        <<"acked_seq">> => AckedSeq,
+        <<"writer_segment">> => WriterSegment
+    })),
+    TmpPath = MetadataPath ++ ".tmp",
+    case file:write_file(TmpPath, Metadata, [write, binary]) of
+        ok ->
+            file:rename(TmpPath, MetadataPath);
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec load_segments(file:filename_all(), pos_integer()) -> {ok, [#segment{}]} | {error, term()}.
+load_segments(Dir, MaxRecordBytes) ->
+    case file:list_dir(Dir) of
+        {ok, Names} ->
+            Ids = lists:sort([Id || Name <- Names, {ok, Id} <- [parse_segment_id(Name)]]),
+            load_segments(Dir, Ids, MaxRecordBytes, []);
+        {error, enoent} ->
+            {ok, []};
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec load_segments(file:filename_all(), [pos_integer()], pos_integer(), [#segment{}]) ->
+    {ok, [#segment{}]} | {error, term()}.
+load_segments(_Dir, [], _MaxRecordBytes, Acc) ->
+    {ok, lists:reverse(Acc)};
+load_segments(Dir, [Id | Rest], MaxRecordBytes, Acc) ->
+    Path = segment_path(Dir, Id),
+    case scan_segment(Path, Id, MaxRecordBytes) of
+        {ok, undefined} ->
+            load_segments(Dir, Rest, MaxRecordBytes, Acc);
+        {ok, Segment} ->
+            load_segments(Dir, Rest, MaxRecordBytes, [Segment | Acc]);
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec parse_segment_id(string()) -> {ok, pos_integer()} | error.
+parse_segment_id(Name) ->
+    case re:run(Name, "^" ++ ?SEGMENT_PREFIX ++ "([0-9]+)\\" ++ ?SEGMENT_EXT ++ "$",
+        [{capture, [1], list}])
+    of
+        {match, [Digits]} ->
+            case list_to_integer(Digits) of
+                Id when Id > 0 ->
+                    {ok, Id};
+                _ ->
+                    error
+            end;
+        nomatch ->
+            error
+    end.
+
+-spec segment_path(file:filename_all(), pos_integer()) -> file:filename_all().
+segment_path(Dir, Id) ->
+    filename:join(Dir, ?SEGMENT_PREFIX ++ integer_to_list(Id) ++ ?SEGMENT_EXT).
+
+-spec open_writer(file:filename_all()) -> {ok, file:fd()} | {error, term()}.
+open_writer(Path) ->
+    file:open(Path, [append, binary]).
+
+-spec writer_segment_id([#segment{}], pos_integer()) -> pos_integer().
+writer_segment_id([], MetaWriterSegment) ->
+    max(1, MetaWriterSegment);
+writer_segment_id(Segments, _MetaWriterSegment) ->
+    (lists:last(Segments))#segment.id.
+
+-spec max_segment_end([#segment{}]) -> non_neg_integer().
+max_segment_end([]) ->
+    0;
+max_segment_end(Segments) ->
+    lists:max([Segment#segment.end_seq || Segment <- Segments]).
+
+-spec normalize_open_outbox(outbox()) -> {ok, outbox()} | {error, term()}.
+normalize_open_outbox(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) when Seq > 0 ->
+    reset_empty_outbox(Outbox);
+normalize_open_outbox(Outbox) ->
+    prune_acked_segments(Outbox).
+
+-spec ensure_writable_segment(outbox()) ->
+    {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}.
+ensure_writable_segment(Outbox = #outbox{segments = Segments, writer_segment = WriterSegment}) ->
+    case find_segment(WriterSegment, Segments) of
+        undefined ->
+            {ok, Outbox};
+        #segment{records = Records} when Records < Outbox#outbox.segment_record_limit ->
+            {ok, Outbox};
+        #segment{} when length(Segments) >= Outbox#outbox.max_segments ->
+            {dropped, capacity_reached, Outbox};
+        #segment{} ->
+            rotate_writer(Outbox, WriterSegment + 1)
+    end.
+
+-spec rotate_writer(outbox(), pos_integer()) -> {ok, outbox()} | {error, term()}.
+rotate_writer(Outbox = #outbox{dir = Dir, fd = Fd}, NewSegment) ->
+    maybe
+        ok ?= file:sync(Fd),
+        {ok, NFd} ?= open_writer(segment_path(Dir, NewSegment)),
+        _ = file:close(Fd),
+        {ok, Outbox#outbox{writer_segment = NewSegment, fd = NFd}}
+    else
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec mark_written(outbox(), pos_integer()) -> outbox().
+mark_written(Outbox = #outbox{
+    dir = Dir,
+    writer_segment = WriterSegment,
+    segments = Segments
+}, Seq) ->
+    Path = segment_path(Dir, WriterSegment),
+    Segment0 = case find_segment(WriterSegment, Segments) of
+        undefined ->
+            #segment{id = WriterSegment, path = Path, start_seq = Seq};
+        Segment ->
+            Segment
+    end,
+    Records = Segment0#segment.records + 1,
+    Segment1 = Segment0#segment{end_seq = Seq, records = Records},
+    Outbox#outbox{
+        segments = replace_segment(Segment1, Segments),
+        next_seq = Seq + 1,
+        write_seq = Seq
+    }.
+
+-spec replace_segment(#segment{}, [#segment{}]) -> [#segment{}].
+replace_segment(Segment, Segments) ->
+    lists:sort(
+        fun(A, B) -> A#segment.id < B#segment.id end,
+        [Segment | [S || S <- Segments, S#segment.id =/= Segment#segment.id]]
+    ).
+
+-spec find_segment(pos_integer(), [#segment{}]) -> #segment{} | undefined.
+find_segment(Id, Segments) ->
+    case [Segment || Segment <- Segments, Segment#segment.id =:= Id] of
+        [Segment] ->
+            Segment;
+        [] ->
+            undefined
+    end.
+
+-spec next_segment([#segment{}], non_neg_integer()) -> #segment{} | undefined.
+next_segment([], _AckedSeq) ->
+    undefined;
+next_segment([Segment = #segment{end_seq = EndSeq} | _Rest], AckedSeq) when EndSeq > AckedSeq ->
+    Segment;
+next_segment([_Segment | Rest], AckedSeq) ->
+    next_segment(Rest, AckedSeq).
+
+-spec maybe_reset_or_prune(outbox()) -> {ok, outbox()} | {error, term()}.
+maybe_reset_or_prune(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) ->
+    reset_empty_outbox(Outbox);
+maybe_reset_or_prune(Outbox) ->
+    prune_acked_segments(Outbox).
+
+-spec prune_acked_segments(outbox()) -> {ok, outbox()} | {error, term()}.
+prune_acked_segments(Outbox = #outbox{segments = Segments, acked_seq = AckedSeq}) ->
+    {DeleteSegments, KeepSegments} = take_acked_segments(Segments, AckedSeq, []),
+    maybe
+        ok ?= delete_files([Segment#segment.path || Segment <- DeleteSegments]),
+        {ok, Outbox#outbox{segments = KeepSegments}}
+    else
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec take_acked_segments([#segment{}], non_neg_integer(), [#segment{}]) ->
+    {[#segment{}], [#segment{}]}.
+take_acked_segments([Segment = #segment{end_seq = EndSeq} | Rest], AckedSeq, Acc)
+  when EndSeq =< AckedSeq ->
+    take_acked_segments(Rest, AckedSeq, [Segment | Acc]);
+take_acked_segments(Segments, _AckedSeq, Acc) ->
+    {lists:reverse(Acc), Segments}.
+
+-spec reset_empty_outbox(outbox()) -> {ok, outbox()} | {error, term()}.
+reset_empty_outbox(Outbox = #outbox{
+    dir = Dir,
+    fd = Fd,
+    writer_segment = WriterSegment,
+    segments = Segments
+}) ->
+    CurrentWriterPath = segment_path(Dir, WriterSegment),
+    SegmentPaths = [Segment#segment.path || Segment <- Segments],
+    Paths = lists:usort([CurrentWriterPath | SegmentPaths]),
+    maybe
+        ok ?= file:close(Fd),
+        ok ?= delete_files(Paths),
+        {ok, NFd} ?= open_writer(segment_path(Dir, 1)),
+        {ok, Outbox#outbox{
+            writer_segment = 1,
+            fd = NFd,
+            segments = [],
+            next_seq = 1,
+            write_seq = 0,
+            acked_seq = 0
+        }}
+    else
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec delete_files([file:filename_all()]) -> ok | {error, term()}.
+delete_files([]) ->
+    ok;
+delete_files([Path | Rest]) ->
+    case file:delete(Path) of
+        ok ->
+            delete_files(Rest);
+        {error, enoent} ->
+            delete_files(Rest);
+        {error, Reason} ->
+            {error, {delete_failed, Path, Reason}}
+    end.
+
+-spec scan_segment(file:filename_all(), pos_integer(), pos_integer()) ->
+    {ok, #segment{} | undefined} | {error, term()}.
+scan_segment(Path, Id, MaxRecordBytes) ->
+    case file:open(Path, [read, binary]) of
+        {ok, Fd} ->
+            Result = scan_segment(Fd, Id, Path, MaxRecordBytes, 0, 0, 0),
+            _ = file:close(Fd),
+            Result;
+        {error, enoent} ->
+            {ok, undefined};
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec scan_segment(file:fd(), pos_integer(), file:filename_all(), pos_integer(),
+    non_neg_integer(), non_neg_integer(), non_neg_integer()) ->
+    {ok, #segment{} | undefined} | {error, term()}.
+scan_segment(Fd, Id, Path, MaxRecordBytes, StartSeq, EndSeq, Records) ->
+    case read_record(Fd, MaxRecordBytes) of
+        eof when Records =:= 0 ->
+            {ok, undefined};
+        eof ->
+            {ok, #segment{
+                id = Id,
+                path = Path,
+                start_seq = StartSeq,
+                end_seq = EndSeq,
+                records = Records
+            }};
+        {ok, Seq, _Packet} ->
+            NStartSeq = case StartSeq of
+                0 -> Seq;
+                _ -> StartSeq
+            end,
+            scan_segment(Fd, Id, Path, MaxRecordBytes, NStartSeq, Seq, Records + 1);
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec encode_record(pos_integer(), binary()) -> binary().
+encode_record(Seq, Packet) ->
+    PacketSize = byte_size(Packet),
+    Magic = ?RECORD_MAGIC,
+    Header = <<Magic/binary, ?RECORD_VERSION:8/unsigned, ?RECORD_HEADER_SIZE:8/unsigned,
+        PacketSize:32/unsigned-big, Seq:64/unsigned-big>>,
+    Crc32 = erlang:crc32(<<Header/binary, Packet/binary>>),
+    <<Header/binary, Packet/binary, Crc32:32/unsigned-big>>.
+
+-spec read_next_unacked(file:fd(), non_neg_integer(), pos_integer()) ->
+    eof | {ok, pos_integer(), binary()} | {error, term()}.
+read_next_unacked(Fd, AckedSeq, MaxRecordBytes) ->
+    case read_record(Fd, MaxRecordBytes) of
+        eof ->
+            eof;
+        {ok, Seq, Packet} when Seq > AckedSeq ->
+            {ok, Seq, Packet};
+        {ok, _Seq, _Packet} ->
+            read_next_unacked(Fd, AckedSeq, MaxRecordBytes);
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec read_record(file:fd(), pos_integer()) -> eof | {ok, pos_integer(), binary()} | {error, term()}.
+read_record(Fd, MaxRecordBytes) ->
+    case find_next_magic(Fd) of
+        eof ->
+            eof;
+        {ok, StartPos} ->
+            case read_record_after_magic(Fd, MaxRecordBytes) of
+                {ok, Seq, Packet} ->
+                    {ok, Seq, Packet};
+                {resync, _Reason} ->
+                    case file:position(Fd, {bof, StartPos + 1}) of
+                        {ok, _} ->
+                            read_record(Fd, MaxRecordBytes);
+                        {error, Reason} ->
+                            {error, Reason}
+                    end
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec find_next_magic(file:fd()) -> eof | {ok, non_neg_integer()} | {error, term()}.
+find_next_magic(Fd) ->
+    case file:position(Fd, cur) of
+        {ok, StartPos} ->
+            case file:read(Fd, ?RECORD_MAGIC_SIZE) of
+                eof ->
+                    eof;
+                {ok, Magic} when byte_size(Magic) < ?RECORD_MAGIC_SIZE ->
+                    eof;
+                {ok, ?RECORD_MAGIC} ->
+                    {ok, StartPos};
+                {ok, Window} ->
+                    find_next_magic(Fd, Window);
+                {error, Reason} ->
+                    {error, Reason}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec find_next_magic(file:fd(), binary()) -> eof | {ok, non_neg_integer()} | {error, term()}.
+find_next_magic(Fd, Window) ->
+    case file:read(Fd, 1) of
+        eof ->
+            eof;
+        {ok, Byte} ->
+            Tail = binary:part(Window, 1, ?RECORD_MAGIC_SIZE - 1),
+            NWindow = <<Tail/binary, Byte/binary>>,
+            case NWindow of
+                ?RECORD_MAGIC ->
+                    case file:position(Fd, cur) of
+                        {ok, Pos} ->
+                            {ok, Pos - ?RECORD_MAGIC_SIZE};
+                        {error, Reason} ->
+                            {error, Reason}
+                    end;
+                _ ->
+                    find_next_magic(Fd, NWindow)
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec read_record_after_magic(file:fd(), pos_integer()) ->
+    {ok, pos_integer(), binary()} | {resync, term()}.
+read_record_after_magic(Fd, MaxRecordBytes) ->
+    HeaderRestSize = ?RECORD_HEADER_SIZE - ?RECORD_MAGIC_SIZE,
+    case file:read(Fd, HeaderRestSize) of
+        {ok, <<Version:8/unsigned, HeaderSize:8/unsigned, PacketSize:32/unsigned-big,
+            Seq:64/unsigned-big>>} ->
+            read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, MaxRecordBytes);
+        {ok, Partial} ->
+            {resync, {truncated_record_header, Partial}};
+        eof ->
+            {resync, truncated_record_header};
+        {error, Reason} ->
+            {resync, Reason}
+    end.
+
+-spec read_record_payload(file:fd(), non_neg_integer(), non_neg_integer(),
+    non_neg_integer(), non_neg_integer(), pos_integer()) ->
+    {ok, pos_integer(), binary()} | {resync, term()}.
+read_record_payload(_Fd, Version, _HeaderSize, _PacketSize, _Seq, _MaxRecordBytes)
+  when Version =/= ?RECORD_VERSION ->
+    {resync, {invalid_record_version, Version}};
+read_record_payload(_Fd, _Version, HeaderSize, _PacketSize, _Seq, _MaxRecordBytes)
+  when HeaderSize =/= ?RECORD_HEADER_SIZE ->
+    {resync, {invalid_record_header_size, HeaderSize}};
+read_record_payload(_Fd, _Version, _HeaderSize, PacketSize, _Seq, MaxRecordBytes)
+  when PacketSize =:= 0; PacketSize > MaxRecordBytes ->
+    {resync, {invalid_record_packet_size, PacketSize}};
+read_record_payload(_Fd, _Version, _HeaderSize, _PacketSize, Seq, _MaxRecordBytes)
+  when Seq =:= 0 ->
+    {resync, {invalid_record_seq, Seq}};
+read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, _MaxRecordBytes) ->
+    case file:read(Fd, PacketSize + ?RECORD_CRC_SIZE) of
+        {ok, <<Packet:PacketSize/binary, Crc32:32/unsigned-big>>} ->
+            validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32);
+        {ok, Partial} ->
+            {resync, {truncated_record_payload, PacketSize, byte_size(Partial)}};
+        eof ->
+            {resync, {truncated_record_payload, PacketSize, 0}};
+        {error, Reason} ->
+            {resync, Reason}
+    end.
+
+-spec validate_record_crc(non_neg_integer(), non_neg_integer(), non_neg_integer(),
+    pos_integer(), binary(), non_neg_integer()) ->
+    {ok, pos_integer(), binary()} | {resync, term()}.
+validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32) ->
+    Magic = ?RECORD_MAGIC,
+    Header = <<Magic/binary, Version:8/unsigned, HeaderSize:8/unsigned,
+        PacketSize:32/unsigned-big, Seq:64/unsigned-big>>,
+    case erlang:crc32(<<Header/binary, Packet/binary>>) of
+        Crc32 ->
+            {ok, Seq, Packet};
+        Expected ->
+            {resync, {crc32_mismatch, Seq, Expected, Crc32}}
+    end.

apps/efka/src/mnesia/cache_model.erl

@@ -1,74 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author aresei
-%%% @copyright (C) 2023, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 04. 7月 2023 12:31
-%%%-------------------------------------------------------------------
--module(cache_model).
--author("aresei").
--include("efka_tables.hrl").
--include_lib("stdlib/include/qlc.hrl").
-
--define(TAB, cache).
-
-%% API
--export([create_table/0]).
--export([insert/2, get_all_cache/0, fetch_next/0, delete/1, next_id/0]).
--export([first_key/0]).
-
-create_table() ->
-    %% id生成器
-    {atomic, ok} = mnesia:create_table(cache, [
-        {attributes, record_info(fields, cache)},
-        {record_name, cache},
-        {disc_copies, [node()]},
-        {type, ordered_set}
-    ]).
-
-next_id() ->
-    id_generator_model:next_id(?TAB).
-
--spec insert(Method :: integer(), Data :: binary()) -> ok | {error, Reason :: any()}.
-insert(Method, Data) when is_integer(Method), is_binary(Data) ->
-    Cache = #cache{id = next_id(), method = Method, data = Data},
-    case mnesia:transaction(fun() -> mnesia:write(?TAB, Cache, write) end) of
-        {'atomic', ok} ->
-            ok;
-        {'aborted', Reason} ->
-            {error, Reason}
-    end.
-
-fetch_next() ->
-    case mnesia:dirty_first(?TAB) of
-        '$end_of_table' ->
-            error;
-        Id ->
-            [Entry] = mnesia:dirty_read(?TAB, Id),
-            {ok, Entry}
-    end.
-
-delete(Id) when is_integer(Id) ->
-    case mnesia:transaction(fun() -> mnesia:delete(?TAB, Id, write) end) of
-        {'atomic', ok} ->
-            ok;
-        {'aborted', Reason} ->
-            {error, Reason}
-    end.
-
--spec get_all_cache() -> [#cache{}].
-get_all_cache() ->
-    Fun = fun() ->
-        Q = qlc:q([E || E <- mnesia:table(?TAB)]),
-        qlc:e(Q)
-          end,
-    case mnesia:transaction(Fun) of
-        {'atomic', Res} ->
-            Res;
-        {'aborted', _} ->
-            []
-    end.
-
-first_key() ->
-    mnesia:dirty_first(?TAB).

apps/efka/src/mnesia/id_generator_model.erl

@@ -1,26 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 06. 5月 2025 10:32
-%%%-------------------------------------------------------------------
--module(id_generator_model).
--author("anlicheng").
--include("efka_tables.hrl").
-
-%% API
--export([create_table/0, next_id/1]).
-
-create_table() ->
-    %% id生成器
-    {atomic, ok} = mnesia:create_table(id_generator, [
-        {attributes, record_info(fields, id_generator)},
-        {record_name, id_generator},
-        {disc_copies, [node()]},
-        {type, ordered_set}
-    ]).
-
-next_id(Tab) when is_atom(Tab) ->
-    mnesia:dirty_update_counter(id_generator, Tab, 1).

apps/efka/src/mnesia/service_model.erl

@@ -1,140 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author aresei
-%%% @copyright (C) 2023, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 04. 7月 2023 12:31
-%%%-------------------------------------------------------------------
--module(service_model).
--author("aresei").
--include("efka_tables.hrl").
--include_lib("stdlib/include/qlc.hrl").
-
--define(TAB, service).
-
-%% API
--export([create_table/0]).
--export([insert/1, get_all_services/0, get_all_service_ids/0, get_running_services/0]).
--export([get_config_json/1, set_config/2, get_service/1, get_status/1, change_status/2]).
--export([display_services/0]).
-
-create_table() ->
-    %% id生成器
-    {atomic, ok} = mnesia:create_table(service, [
-        {attributes, record_info(fields, service)},
-        {record_name, service},
-        {disc_copies, [node()]},
-        {type, ordered_set}
-    ]).
-
-insert(Service = #service{}) ->
-    case mnesia:transaction(fun() -> mnesia:write(?TAB, Service, write) end) of
-        {'atomic', Res} ->
-            Res;
-        {'aborted', Reason} ->
-            {error, Reason}
-    end.
-
-change_status(ServiceId, NewStatus) when is_binary(ServiceId), is_integer(NewStatus) ->
-    Fun = fun() ->
-        case mnesia:read(?TAB, ServiceId, write) of
-            [] ->
-                mnesia:abort(<<"service not found">>);
-            [Service] ->
-                mnesia:write(?TAB, Service#service{status = NewStatus}, write)
-        end
-          end,
-    case mnesia:transaction(Fun) of
-        {'atomic', ok} ->
-            ok;
-        {'aborted', Reason} ->
-            {error, Reason}
-    end.
-
--spec set_config(ServiceId :: binary(), ConfigJson :: binary()) -> ok | {error, Reason :: any()}.
-set_config(ServiceId, ConfigJson) when is_binary(ServiceId), is_binary(ConfigJson) ->
-    Fun = fun() ->
-        case mnesia:read(?TAB, ServiceId, write) of
-            [] ->
-                mnesia:abort(<<"service not found">>);
-            [S] ->
-                mnesia:write(?TAB, S#service{config_json = ConfigJson}, write)
-        end
-          end,
-    case mnesia:transaction(Fun) of
-        {'atomic', ok} ->
-            ok;
-        {'aborted', Reason} ->
-            {error, Reason}
-    end.
-
--spec get_config_json(ServiceId :: binary()) -> error | {ok, ConfigJson :: binary()}.
-get_config_json(ServiceId) when is_binary(ServiceId) ->
-    case mnesia:dirty_read(?TAB, ServiceId) of
-        [] ->
-            error;
-        [#service{config_json = ConfigJson}] ->
-            {ok, ConfigJson}
-    end.
-
--spec get_status(ServiceId :: binary()) -> Status :: integer().
-get_status(ServiceId) when is_binary(ServiceId) ->
-    case mnesia:dirty_read(?TAB, ServiceId) of
-        [] ->
-            0;
-        [#service{status = Status}] ->
-            Status
-    end.
-
--spec get_service(ServiceId :: binary()) -> error | {ok, Service :: #service{}}.
-get_service(ServiceId) when is_binary(ServiceId) ->
-    case mnesia:dirty_read(?TAB, ServiceId) of
-        [] ->
-            error;
-        [Service] ->
-            {ok, Service}
-    end.
-
--spec get_all_services() -> [#service{}].
-get_all_services() ->
-    Fun = fun() ->
-        Q = qlc:q([E || E <- mnesia:table(?TAB)]),
-        qlc:e(Q)
-          end,
-
-    case mnesia:transaction(Fun) of
-        {'atomic', Res} ->
-            Res;
-        {'aborted', _} ->
-            []
-    end.
-
--spec get_all_service_ids() -> [ServiceId :: binary()].
-get_all_service_ids() ->
-    mnesia:dirty_all_keys(?TAB).
-
--spec get_running_services() -> {ok, [#service{}]} | {error, Reason :: term()}.
-get_running_services() ->
-    F = fun() ->
-        Q = qlc:q([E || E <- mnesia:table(?TAB), E#service.status == 1]),
-        qlc:e(Q)
-        end,
-    case mnesia:transaction(F) of
-        {atomic, Services} ->
-            {ok, Services};
-        {aborted, Error} ->
-            {error, Error}
-    end.
-
-display_services() ->
-    F = fun() ->
-        Q = qlc:q([E || E <- mnesia:table(?TAB)]),
-        qlc:e(Q)
-        end,
-    case mnesia:transaction(F) of
-        {atomic, Services} ->
-            {ok, Services};
-        {aborted, Error} ->
-            {error, Error}
-    end.

apps/efka/src/mnesia/task_log_model.erl

@@ -1,46 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author aresei
-%%% @copyright (C) 2023, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 04. 7月 2023 12:31
-%%%-------------------------------------------------------------------
--module(task_log_model).
--author("aresei").
--include("efka_tables.hrl").
--include_lib("stdlib/include/qlc.hrl").
-
--define(TAB, task_log).
-
-%% API
--export([create_table/0]).
--export([insert/2, get_logs/1]).
-
-create_table() ->
-    %% id生成器
-    {atomic, ok} = mnesia:create_table(task_log, [
-        {attributes, record_info(fields, task_log)},
-        {record_name, task_log},
-        {disc_copies, [node()]},
-        {type, ordered_set}
-    ]).
-
--spec insert(TaskId :: integer(), Logs :: [binary()]) -> ok | {error, Reason :: term()}.
-insert(TaskId, Logs) when is_integer(TaskId), is_list(Logs) ->
-    TaskLog = #task_log{task_id = TaskId, logs = Logs},
-    case mnesia:transaction(fun() -> mnesia:write(?TAB, TaskLog, write) end) of
-        {'atomic', Res} ->
-            Res;
-        {'aborted', Reason} ->
-            {error, Reason}
-    end.
-
--spec get_logs(TaskId :: integer()) -> Logs :: [binary()].
-get_logs(TaskId) when is_integer(TaskId) ->
-    case mnesia:dirty_read(?TAB, TaskId) of
-        [] ->
-            [];
-        [#task_log{logs = Logs}] ->
-            Logs
-    end.

apps/efka/src/service/efka_service.erl

@@ -0,0 +1,145 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%% 1. 需要管理服务的整个生命周期，包括: 启动，停止
+%%% 2. 需要监控服务的状态，通过port的方式
+%%% 3. 服务的启动和关闭，需要在更高的层级控制
+%%% @end
+%%% Created : 18. 4月 2025 16:50
+%%%-------------------------------------------------------------------
+-module(efka_service).
+-author("anlicheng").
+-include("efka_tables.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/2]).
+-export([get_name/1, get_pid/1, attach_channel/2]).
+-export([metric_data/3, send_event/3]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-record(state, {
+    service_id :: binary(),
+    %% 通道id信息
+    channel_pid :: pid() | undefined
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec get_name(ServiceId :: binary()) -> atom().
+get_name(ServiceId) when is_binary(ServiceId) ->
+    list_to_atom("efka_service:" ++ binary_to_list(ServiceId)).
+
+-spec get_pid(ServiceId :: binary()) -> undefined | pid().
+get_pid(ServiceId) when is_binary(ServiceId) ->
+    whereis(get_name(ServiceId)).
+
+-spec metric_data(Pid :: pid(), RouteKey :: binary(), Metric :: binary()) -> ok.
+metric_data(Pid, RouteKey, Metric) when is_pid(Pid), is_binary(RouteKey),  is_binary(Metric) ->
+    gen_server:cast(Pid, {metric_data, RouteKey, Metric}).
+
+-spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> ok.
+send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) ->
+    gen_server:cast(Pid, {send_event, EventType, Params}).
+
+-spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}.
+attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) ->
+    gen_server:call(Pid, {attach_channel, ChannelPid}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link(Name :: atom(), Service :: binary()) ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) ->
+    gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([ServiceId]) ->
+    %% supervisor进程通过exit(ChildPid, shutdown)调用的时候，确保terminate函数被调用
+    logger:debug("[efka_service] service_id: ~p, started", [ServiceId]),
+    {ok, #state{service_id = ServiceId}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 绑定channel
+handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) ->
+    case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of
+        false ->
+            erlang:monitor(process, ChannelPid),
+            logger:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
+            {reply, ok, State#state{channel_pid = ChannelPid}};
+        true ->
+            {reply, {error, <<"channel exists">>}, State}
+    end;
+
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast({metric_data, RouteKey, Metric}, State = #state{service_id = ServiceId}) ->
+    logger:debug("[efka_service] metric_data service_id: ~p, route_key: ~p, metric data: ~p", [ServiceId, RouteKey, Metric]),
+    efka_iot_client:metric_data(RouteKey, Metric),
+    {noreply, State};
+
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 处理channel进程的退出
+handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) ->
+    logger:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
+    {noreply, State#state{channel_pid = undefined}}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(Reason, _State = #state{service_id = ServiceId}) ->
+    logger:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================

apps/efka/src/service/efka_service_channel.erl

@@ -0,0 +1,169 @@
+%%%-------------------------------------------------------------------
+%%% @author licheng5
+%%% @copyright (C) 2021, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 11. 1月 2021 上午12:17
+%%%-------------------------------------------------------------------
+-module(efka_service_channel).
+-author("licheng5").
+-include("efka_tables.hrl").
+-include("efka_service_pb.hrl").
+
+%% 一级帧类型
+%% REQUEST: 需要响应的请求帧
+%% RESPONSE: 对 REQUEST 的响应帧
+%% CAST: 单向消息帧，不需要响应
+-define(FRAME_REQUEST, 16#01).
+-define(FRAME_REPLY, 16#02).
+-define(FRAME_CAST, 16#03).
+
+%% API
+-export([init/2]).
+-export([websocket_init/1, websocket_handle/2, websocket_info/2, terminate/3]).
+
+-record(state, {
+    service_id :: undefined | binary(),
+    service_pid :: undefined | pid(),
+    subscribed_topics = sets:new(),
+    is_registered = false :: boolean()
+}).
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% 逻辑处理方法
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+-spec init(term(), term()) -> {cowboy_websocket, term(), term()}.
+init(Req, Opts) ->
+    {cowboy_websocket, Req, Opts}.
+
+-spec websocket_init(term()) -> {ok, #state{}}.
+websocket_init(_State) ->
+    logger:debug("[efka_service_channel] get a new connection"),
+    %% 初始状态为true
+    {ok, #state{}}.
+
+-spec websocket_handle(term(), #state{}) ->
+    {reply, term(), #state{}} | {ok, #state{}}.
+websocket_handle(ping, State) ->
+    {reply, pong, State};
+
+websocket_handle({binary, <<?FRAME_REQUEST, PacketBin/binary>>}, State) ->
+    Request = efka_service_pb:decode_msg(PacketBin, 'ServiceRequest'),
+    logger:debug("[efka_service_channel] get request: ~p", [Request]),
+    handle_request(Request, State);
+websocket_handle({binary, <<?FRAME_CAST, PacketBin/binary>>}, State) ->
+    Cast = efka_service_pb:decode_msg(PacketBin, 'ServiceCast'),
+    logger:debug("[efka_service_channel] get cast: ~p", [Cast]),
+    handle_cast(Cast, State);
+
+websocket_handle(Info, State) ->
+    logger:error("[efka_service_channel] get a unknown message: ~p, channel will closed", [Info]),
+    {ok, State}.
+
+%% 订阅的消息
+-spec websocket_info(term(), #state{}) ->
+    {reply, term(), #state{}} | {stop, #state{}} | {ok, #state{}}.
+websocket_info({topic_broadcast, Topic, Content}, State = #state{}) ->
+    Packet = efka_service_pb:encode_msg(#'ServiceCast'{
+        body = {topic_event, #'ServiceCast.TopicEvent'{topic = Topic, content = Content}}
+    }),
+    logger:debug("[efka_service_channel] will publish topic: ~p", [Topic]),
+    {reply, {binary, <<?FRAME_CAST, Packet/binary>>}, State};
+
+%% service进程关闭
+websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) ->
+    logger:debug("[efka_service_channel] container_pid: ~p, exited: ~p", [ServicePid, Reason]),
+    {stop, State#state{service_pid = undefined}};
+
+%% 处理关闭信号
+websocket_info({stop, Reason}, State) ->
+    logger:debug("[efka_service_channel] the channel will be closed with reason: ~p", [Reason]),
+    {stop, State};
+
+%% 处理其他未知消息
+websocket_info(Info, State) ->
+    logger:debug("[efka_service_channel] channel get unknown info: ~p", [Info]),
+    {ok, State}.
+
+%% 进程关闭事件
+-spec terminate(term(), term(), #state{}) -> ok.
+terminate(Reason, _Req, State = #state{service_id = ServiceId, is_registered = IsRegistered}) ->
+    ok = efka_subscription:unsubscribe_all(self()),
+    case IsRegistered of
+        true ->
+            ok = efka_service_model:change_status(ServiceId, 0);
+        false ->
+            ok
+    end,
+    logger:debug("[efka_service_channel] channel close with reason: ~p, state is: ~p", [Reason, State]),
+    ok.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% helper methods
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+%% 注册, 要建立程序和容器之间的关系
+-spec handle_request(efka_service_pb:'ServiceRequest'(), #state{}) -> {reply, {binary, binary()}, #state{}}.
+handle_request(#'ServiceRequest'{packet_id = PacketId, request = {register, #'ServiceRequest.Register'{service_id = ServiceId}}}, State) ->
+    {ok, ServicePid} = efka_service_sup:start_service(ServiceId),
+    case efka_service:attach_channel(ServicePid, self()) of
+        ok ->
+            erlang:monitor(process, ServicePid),
+
+            %% 更新微服务的状态
+            ok = efka_service_model:insert(#service{
+                service_id = ServiceId,
+                container_name = <<>>,
+                status = ?SERVICE_RUNNING,
+                meta_data = #{},
+                create_ts = efka_util:timestamp(),
+                update_ts = efka_util:timestamp()
+            }),
+
+            {reply, {binary, result_reply_packet(PacketId, <<"ok">>)},
+                State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}};
+        {error, Error} ->
+            logger:warning("[efka_service_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]),
+            {reply, {binary, error_reply_packet(PacketId, -1, <<"attach channel failed">>)}, State}
+    end;
+
+%% 订阅事件
+handle_request(#'ServiceRequest'{packet_id = PacketId, request = {subscribe, #'ServiceRequest.Subscribe'{topic = Topic}}},
+    State = #state{subscribed_topics = SubscribedTopics, is_registered = true}) ->
+
+    case efka_subscription:subscribe(Topic, self()) of
+        ok ->
+            Packet = result_reply_packet(PacketId, <<"ok">>),
+            {reply, {binary, Packet}, State#state{subscribed_topics = sets:add_element(Topic, SubscribedTopics)}};
+        {error, Reason} ->
+            Packet = error_reply_packet(PacketId, -1, Reason),
+            {reply, {binary, Packet}, State}
+    end;
+handle_request(#'ServiceRequest'{packet_id = PacketId}, State) ->
+    {reply, {binary, error_reply_packet(PacketId, -1, <<"invalid request">>)}, State}.
+
+-spec handle_cast(efka_service_pb:'ServiceCast'(), #state{}) -> {ok, #state{}}.
+handle_cast(#'ServiceCast'{body = {metric_data, #'ServiceCast.MetricData'{route_key = RouteKey, metric = Metric}}},
+    State = #state{service_pid = ServicePid, is_registered = true}) ->
+    efka_service:metric_data(ServicePid, RouteKey, Metric),
+    {ok, State};
+handle_cast(#'ServiceCast'{body = _Body}, State) ->
+    {ok, State}.
+
+-spec result_reply_packet(integer(), binary()) -> binary().
+result_reply_packet(PacketId, Result) when is_integer(PacketId), is_binary(Result) ->
+    Reply = efka_service_pb:encode_msg(#'ServiceReply'{
+        packet_id = PacketId,
+        reply = {result, Result}
+    }),
+    <<?FRAME_REPLY, Reply/binary>>.
+
+-spec error_reply_packet(integer(), integer(), binary()) -> binary().
+error_reply_packet(PacketId, Code, Message) when is_integer(PacketId), is_integer(Code), is_binary(Message) ->
+    Reply = efka_service_pb:encode_msg(#'ServiceReply'{
+        packet_id = PacketId,
+        reply = {error, #'ServiceReply.Error'{code = Code, message = Message}}
+    }),
+    <<?FRAME_REPLY, Reply/binary>>.

apps/efka/src/service/efka_service_model.erl

@@ -6,7 +6,7 @@
 %%% @end
 %%% Created : 13. 8月 2025 16:41
 %%%-------------------------------------------------------------------
--module(service_model).
+-module(efka_service_model).
 -author("anlicheng").
 -include("efka_tables.hrl").
 
@@ -14,6 +14,7 @@
 
 %% API
 -export([start_link/0]).
+-export([insert/1, change_status/2, get_status/1, get_service/1, get_all_services/0, get_running_services/0]).
 
 %% gen_server callbacks
 -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
@@ -29,20 +30,14 @@
 %%% API
 %%%===================================================================
 
+-spec insert(#service{}) -> ok.
 insert(Service = #service{}) ->
     gen_server:call(?SERVER, {insert, Service}).
 
+-spec change_status(binary(), integer()) -> ok | {error, binary()}.
 change_status(ServiceId, NewStatus) when is_binary(ServiceId), is_integer(NewStatus) ->
     gen_server:call(?SERVER, {change_status, ServiceId, NewStatus}).
 
--spec set_config(ServiceId :: binary(), ConfigJson :: binary()) -> ok | {error, Reason :: any()}.
-set_config(ServiceId, ConfigJson) when is_binary(ServiceId), is_binary(ConfigJson) ->
-    gen_server:call(?SERVER, {set_config, ServiceId, ConfigJson}).
-
--spec get_config_json(ServiceId :: binary()) -> error | {ok, ConfigJson :: binary()}.
-get_config_json(ServiceId) when is_binary(ServiceId) ->
-    gen_server:call(?SERVER, {get_config_json, ServiceId}).
-
 -spec get_status(ServiceId :: binary()) -> Status :: integer().
 get_status(ServiceId) when is_binary(ServiceId) ->
     gen_server:call(?SERVER, {get_status, ServiceId}).
@@ -55,11 +50,7 @@ get_service(ServiceId) when is_binary(ServiceId) ->
 get_all_services() ->
     gen_server:call(?SERVER, get_all_services).
 
--spec get_all_service_ids() -> [ServiceId :: binary()].
-get_all_service_ids() ->
-    gen_server:call(?SERVER, get_all_service_ids).
-
--spec get_running_services() -> {ok, [#service{}]} | {error, Reason :: term()}.
+-spec get_running_services() -> {ok, [#service{}]}.
 get_running_services() ->
     gen_server:call(?SERVER, get_running_services).
 
@@ -81,7 +72,7 @@ start_link() ->
 init([]) ->
     {ok, DetsDir} = application:get_env(efka, dets_dir),
     File = DetsDir ++ "service.dets",
-    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, bag}, {keypos, 2}]),
+    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, set}, {keypos, 2}]),
     {ok, #state{}}.
 
 %% @private
@@ -94,8 +85,18 @@ init([]) ->
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_call({insert, Service}, _From, State = #state{}) ->
-    ok = dets:insert(?TAB, Service),
+handle_call({insert, Service = #service{service_id = ServiceId}}, _From, State = #state{}) ->
+    case dets:lookup(?TAB, ServiceId) of
+        [] ->
+            ok = dets:insert(?TAB, Service);
+        [OldService] ->
+            NewService = OldService#service{
+                meta_data = Service#service.meta_data,
+                container_name = Service#service.container_name,
+                update_ts = Service#service.update_ts
+            },
+            ok = dets:insert(?TAB, NewService)
+    end,
     {reply, ok, State};
 
 handle_call({change_status, ServiceId, NewStatus}, _From, State = #state{}) ->
@@ -108,24 +109,6 @@ handle_call({change_status, ServiceId, NewStatus}, _From, State = #state{}) ->
             {reply, ok, State}
     end;
 
-handle_call({set_config, ServiceId, ConfigJson}, _From, State = #state{}) ->
-    case dets:lookup(?TAB, ServiceId) of
-        [] ->
-            {reply, {error, <<"service not found">>}, State};
-        [OldService] ->
-            NewService = OldService#service{config_json = ConfigJson},
-            ok = dets:insert(?TAB, NewService),
-            {reply, ok, State}
-    end;
-
-handle_call({get_config_json, ServiceId}, _From, State = #state{}) ->
-    case dets:lookup(?TAB, ServiceId) of
-        [] ->
-            {reply, error, State};
-        [#service{config_json = ConfigJson}] ->
-            {reply, {ok, ConfigJson}, State}
-    end;
-
 handle_call({get_status, ServiceId}, _From, State = #state{}) ->
     case dets:lookup(?TAB, ServiceId) of
         [] ->
@@ -141,7 +124,7 @@ handle_call(get_all_services, _From, State = #state{}) ->
 handle_call(get_running_services, _From, State = #state{}) ->
     Items = dets:foldl(fun(Record, Acc) -> [Record|Acc] end, [], ?TAB),
     RunningItems = lists:filter(fun(#service{status = Status}) -> Status =:= 1 end, lists:reverse(Items)),
-    {reply, RunningItems, State};
+    {reply, {ok, RunningItems}, State};
 
 handle_call(_Request, _From, State = #state{}) ->
     {reply, ok, State}.

apps/efka/src/service/efka_service_sup.erl

@@ -39,16 +39,10 @@ start_link() ->
 %% this function is called by the new process to find out about
 %% restart strategy, maximum restart frequency and child
 %% specifications.
+-spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
 init([]) ->
     SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
-    %% 简化逻辑，只启动需要运行的微服务
-    {ok, Services} = service_model:get_running_services(),
-    ServiceIds = lists:map(fun(#service{service_id = ServiceId}) -> ServiceId end, Services),
-    lager:debug("[efka_service_sup] will start services: ~p", [ServiceIds]),
-
-    Specs = lists:map(fun(ServiceId) -> child_spec(ServiceId) end, Services),
-
-    {ok, {SupFlags, Specs}}.
+    {ok, {SupFlags, []}}.
 
 %%%===================================================================
 %%% Internal functions
@@ -71,8 +65,7 @@ stop_service(ServiceId) when is_binary(ServiceId) ->
     supervisor:terminate_child(?MODULE, ChildId),
     supervisor:delete_child(?MODULE, ChildId).
 
-child_spec(#service{service_id = ServiceId}) when is_binary(ServiceId) ->
-    child_spec(ServiceId);
+-spec child_spec(binary()) -> supervisor:child_spec().
 child_spec(ServiceId) when is_binary(ServiceId) ->
     Name = efka_service:get_name(ServiceId),
     #{
@@ -82,4 +75,4 @@ child_spec(ServiceId) when is_binary(ServiceId) ->
         shutdown => 5000,
         type => worker,
         modules => ['efka_service']
-    }.
+    }.

apps/efka/src/service/efka_subscription.erl

@@ -13,7 +13,7 @@
 
 %% API
 -export([start_link/0]).
--export([subscribe/2, publish/2]).
+-export([subscribe/2, unsubscribe_all/1, publish/3, debug_info/0]).
 -export([match_components/2, is_valid_components/1, of_components/1]).
 
 %% gen_server callbacks
@@ -34,20 +34,30 @@
 }).
 
 -record(state, {
-    subscribers = []
+    subscribers = [],
+    %% qos未1，并且未被消费的消息
+    remaining_messages = []
 }).
 
 %%%===================================================================
 %%% API
 %%%===================================================================
 
--spec subscribe(Topic :: binary(), SubscriberPid :: pid()) -> no_return().
+-spec subscribe(Topic :: binary(), SubscriberPid :: pid()) -> ok | {error, Reason :: binary()}.
 subscribe(Topic, SubscriberPid) when is_binary(Topic), is_pid(SubscriberPid) ->
-    gen_server:cast(?SERVER, {subscribe, Topic, SubscriberPid}).
+    gen_server:call(?SERVER, {subscribe, Topic, SubscriberPid}).
 
--spec publish(Topic :: binary(), Content :: binary()) -> no_return().
-publish(Topic, Content) when is_binary(Topic), is_binary(Content) ->
-    gen_server:cast(?SERVER, {publish, Topic, Content}).
+-spec publish(Topic :: binary(), Qos :: integer(), Content :: binary()) -> ok.
+publish(Topic, Qos, Content) when is_binary(Topic), is_integer(Qos), is_binary(Content) ->
+    gen_server:cast(?SERVER, {publish, Topic, Qos, Content}).
+
+-spec unsubscribe_all(pid()) -> ok.
+unsubscribe_all(SubscriberPid) when is_pid(SubscriberPid) ->
+    gen_server:call(?SERVER, {unsubscribe_all, SubscriberPid}).
+
+-spec debug_info() -> {ok, Info :: map()}.
+debug_info() ->
+    gen_server:call(?SERVER, debug_info).
 
 %% @doc Spawns the server and registers the local name (unique)
 -spec(start_link() ->
@@ -77,8 +87,38 @@ init([]) ->
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
+%%  同一个SubscriberPid只能订阅同一个topic一次
+handle_call({subscribe, Topic, SubscriberPid}, _From, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
+    Components = of_components(Topic),
+    case is_valid_components(Components) of
+        true ->
+            case has_subscription(Topic, SubscriberPid, Subscribers) of
+                true ->
+                    {reply, ok, State};
+                false ->
+                    Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)},
+                    %% 只有首次看到该pid时才建立monitor，避免重复monitor
+                    case has_subscriber_pid(SubscriberPid, Subscribers) of
+                        true ->
+                            ok;
+                        false ->
+                            erlang:monitor(process, SubscriberPid)
+                    end,
+                    %% 处理遗留的消息
+                    RestRemainingMessages = dispatch_remaining_messages(Sub, RemainingMessages),
+                    {reply, ok, State#state{subscribers = Subscribers ++ [Sub], remaining_messages = RestRemainingMessages}}
+            end;
+        false ->
+            {reply, {error, <<"invalid topic name">>}, State}
+    end;
+handle_call({unsubscribe_all, SubscriberPid}, _From, State = #state{subscribers = Subscribers}) ->
+    {reply, ok, State#state{subscribers = remove_subscriber_pid(SubscriberPid, Subscribers)}};
+handle_call(debug_info, _From, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
+    Info = #{
+        subscribes => Subscribers,
+        remaining_messages => RemainingMessages
+    },
+    {reply, {ok, Info}, State}.
 
 %% @private
 %% @doc Handling cast messages
@@ -86,29 +126,19 @@ handle_call(_Request, _From, State = #state{}) ->
     {noreply, NewState :: #state{}} |
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
-%%  同一个SubscriberPid只能订阅同一个topic一次
-handle_cast({subscribe, Topic, SubscriberPid}, State = #state{subscribers = Subscribers}) ->
-    Components = of_components(Topic),
-    case is_valid_components(Components) of
-        true ->
-            Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)},
-            %% 建立到SubscriberPid的monitor，进程退出需要清理订阅
-            erlang:monitor(process, SubscriberPid),
-
-            {noreply, State#state{subscribers = Subscribers ++ [Sub]}};
-        false ->
-            {noreply, State}
-    end;
-
 %% 发布消息
-handle_cast({publish, Topic, Content}, State = #state{subscribers = Subscribers}) ->
+handle_cast({publish, Topic, Qos, Content}, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
     MatchedSubscribers = match_subscribers(Subscribers, Topic),
-    lists:foreach(fun(#subscriber{subscriber_pid = SubscriberPid}) ->
-        SubscriberPid ! {topic_broadcast, Topic, Content}
-                  end, MatchedSubscribers),
-
-    lager:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]),
-    {noreply, State}.
+    logger:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]),
+    case MatchedSubscribers of
+        [_|_] ->
+            broadcast(Topic, Content, MatchedSubscribers),
+            {noreply, State};
+        [] when Qos =:= 0 ->
+            {noreply, State};
+        [] ->
+            {noreply, State#state{remaining_messages = [{Topic, Content}|RemainingMessages]}}
+    end.
 
 %% @private
 %% @doc Handling all non call/cast messages
@@ -117,12 +147,12 @@ handle_cast({publish, Topic, Content}, State = #state{subscribers = Subscribers}
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
 handle_info({'DOWN', _Ref, process, SubscriberPid, Reason}, State = #state{subscribers = Subscribers}) ->
-    lager:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]),
-    NSubscribers = lists:filter(fun(#subscriber{subscriber_pid = Pid0}) -> SubscriberPid /= Pid0 end, Subscribers),
+    logger:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]),
+    NSubscribers = remove_subscriber_pid(SubscriberPid, Subscribers),
     {noreply, State#state{subscribers = NSubscribers}};
 
 handle_info(Info, State = #state{}) ->
-    lager:debug("[efka_subscription] get unknown info: ~p", [Info]),
+    logger:debug("[efka_subscription] get unknown info: ~p", [Info]),
     {noreply, State}.
 
 %% @private
@@ -164,6 +194,17 @@ match_subscribers(Subscribers, Topic) when is_list(Subscribers), is_binary(Topic
 contain_channel(Pid, Subscribers) when is_pid(Pid), is_list(Subscribers) ->
     lists:search(fun(#subscriber{subscriber_pid = Pid0}) -> Pid == Pid0 end, Subscribers) /= false.
 
+-spec has_subscriber_pid(pid(), [#subscriber{}]) -> boolean().
+has_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) ->
+    lists:any(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =:= SubscriberPid0 end, Subscribers).
+
+-spec has_subscription(binary(), pid(), [#subscriber{}]) -> boolean().
+has_subscription(Topic, SubscriberPid, Subscribers)
+  when is_binary(Topic), is_pid(SubscriberPid), is_list(Subscribers) ->
+    lists:any(fun(#subscriber{topic = Topic0, subscriber_pid = SubscriberPid0}) ->
+        Topic =:= Topic0 andalso SubscriberPid =:= SubscriberPid0
+    end, Subscribers).
+
 %% 开始对比订阅的topic和发布的topic的Components信息
 %% *表示单级匹配，+表示多级匹配；+只能出现一次，并且只能在末尾
 -spec match_components(list(), list()) -> boolean().
@@ -184,6 +225,7 @@ match_components(_, _, _) ->
 of_components(Topic) when is_binary(Topic) ->
     binary:split(Topic, <<$/>>, [global]).
 
+-spec is_valid_components([binary()]) -> boolean().
 is_valid_components([]) ->
     true;
 is_valid_components([<<$+>>|T]) ->
@@ -201,4 +243,28 @@ order_num([<<$*>>|_]) ->
 order_num([<<$+>>|_]) ->
     3;
 order_num([_|Tail]) ->
-    order_num(Tail).
+    order_num(Tail).
+
+-spec broadcast(binary(), binary(), [#subscriber{}]) -> ok.
+broadcast(Topic, Content, MatchedSubscribers) ->
+    lists:foreach(fun(#subscriber{subscriber_pid = SubscriberPid}) ->
+        SubscriberPid ! {topic_broadcast, Topic, Content}
+                  end, MatchedSubscribers).
+
+-spec remove_subscriber_pid(pid(), [#subscriber{}]) -> [#subscriber{}].
+remove_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) ->
+    lists:filter(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =/= SubscriberPid0 end, Subscribers).
+
+-spec dispatch_remaining_messages(Subscriber :: #subscriber{}, RemainingMessages :: list()) -> RestRemainingMessages :: list().
+dispatch_remaining_messages(#subscriber{subscriber_pid = SubscriberPid, components = Components}, RemainingMessages) when is_list(RemainingMessages) ->
+    %% 处理遗留的消息
+    lists:foldl(fun({Topic0, Content0}, Acc) ->
+        Components0 = of_components(Topic0),
+        case match_components(Components, Components0) of
+            true ->
+                SubscriberPid ! {topic_broadcast, Topic0, Content0},
+                Acc;
+            false ->
+                [{Topic0, Content0}|Acc]
+        end
+                end, [], RemainingMessages).

apps/efka/src/tests/efka_iot_outbox_test.erl

@@ -0,0 +1,315 @@
+%%%-------------------------------------------------------------------
+%%% @doc
+%%% efka_iot_outbox 的手动模拟测试模块。
+%%%
+%%% 该模块基于 gen_server 实现，用两个定时 tick 模拟一边写入、一边读取：
+%%% - write_tick 每次向 efka_iot_outbox 写入一条 packet。
+%%% - read_tick 每次从 efka_iot_outbox 读取并 ack 一条 packet。
+%%% - 默认 read_interval = write_interval * 2，因此读取速度是写入速度的
+%%%   50%。
+%%% - 读取到的数据会通过 logger:debug/2 输出。
+%%%
+%%% 默认使用独立临时 dir，避免污染真实运行目录。也可以通过 start_link/1
+%%% 传入 {dir, Dir} 指定 outbox 目录。
+%%%
+%%% 示例：
+%%% <pre>
+%%% efka_iot_outbox_test:start_link([
+%%%     {dir, "/private/tmp/efka_iot_outbox_test"},
+%%%     {write_interval, 100},
+%%%     {read_interval, 200},
+%%%     {max_writes, 1000}
+%%% ]).
+%%% efka_iot_outbox_test:stats().
+%%% efka_iot_outbox_test:stop().
+%%% </pre>
+%%% @end
+%%%-------------------------------------------------------------------
+-module(efka_iot_outbox_test).
+
+-behaviour(gen_server).
+
+-export([start/0, start/1, start_link/0, start_link/1, stop/0, stats/0]).
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+-export([test/0]).
+
+-define(SERVER, ?MODULE).
+-define(DEFAULT_WRITE_INTERVAL, 100).
+-define(DEFAULT_SEGMENT_RECORD_LIMIT, 2000).
+-define(DEFAULT_MAX_SEGMENTS, 5).
+-define(DEFAULT_MAX_RECORD_BYTES, 16 * 1024 * 1024).
+
+-record(state, {
+    outbox :: efka_iot_outbox:outbox(),
+    dir :: file:filename_all(),
+    write_interval = ?DEFAULT_WRITE_INTERVAL :: pos_integer(),
+    read_interval = ?DEFAULT_WRITE_INTERVAL * 2 :: pos_integer(),
+    max_writes = infinity :: pos_integer() | infinity,
+    write_timer = undefined :: undefined | reference(),
+    read_timer = undefined :: undefined | reference(),
+    write_count = 0 :: non_neg_integer(),
+    appended_count = 0 :: non_neg_integer(),
+    read_count = 0 :: non_neg_integer(),
+    dropped_count = 0 :: non_neg_integer(),
+    error_count = 0 :: non_neg_integer()
+}).
+
+-type option() ::
+    {dir, file:filename_all()} |
+    {write_interval, pos_integer()} |
+    {read_interval, pos_integer()} |
+    {max_writes, pos_integer() | infinity} |
+    {segment_record_limit, pos_integer()} |
+    {max_segments, pos_integer()} |
+    {max_record_bytes, pos_integer()}.
+
+
+test() ->
+    efka_iot_outbox_test:start_link([
+        {write_interval, 10},
+        {read_interval, 20},
+        {max_writes, 100_0000}
+    ]).
+
+-spec start() -> {ok, pid()} | {error, term()}.
+start() ->
+    start([]).
+
+-spec start([option()] | map()) -> {ok, pid()} | {error, term()}.
+start(Options) ->
+    gen_server:start({local, ?SERVER}, ?MODULE, Options, []).
+
+-spec start_link() -> {ok, pid()} | {error, term()}.
+start_link() ->
+    start_link([]).
+
+-spec start_link([option()] | map()) -> {ok, pid()} | {error, term()}.
+start_link(Options) ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, Options, []).
+
+-spec stop() -> ok.
+stop() ->
+    gen_server:stop(?SERVER).
+
+-spec stats() -> map().
+stats() ->
+    gen_server:call(?SERVER, stats).
+
+-spec init([option()] | map()) -> {ok, #state{}} | {stop, term()}.
+init(Options0) ->
+    process_flag(trap_exit, true),
+    Options = normalize_options(Options0),
+    WriteInterval = positive_option(write_interval, Options, ?DEFAULT_WRITE_INTERVAL),
+    ReadInterval = positive_option(read_interval, Options, WriteInterval * 2),
+    SegmentRecordLimit = positive_option(segment_record_limit, Options, ?DEFAULT_SEGMENT_RECORD_LIMIT),
+    MaxSegments = positive_option(max_segments, Options, ?DEFAULT_MAX_SEGMENTS),
+    MaxRecordBytes = positive_option(max_record_bytes, Options, ?DEFAULT_MAX_RECORD_BYTES),
+    MaxWrites = max_writes_option(Options),
+
+    Dir = "/private/tmp/efka_outbox/",
+    ok = filelib:ensure_dir(filename:join(Dir, "dummy")),
+    OutboxOptions = #{
+        dir => Dir,
+        segment_record_limit => SegmentRecordLimit,
+        max_segments => MaxSegments,
+        max_record_bytes => MaxRecordBytes
+    },
+    case efka_iot_outbox:open(OutboxOptions) of
+        {ok, Outbox} ->
+            State0 = #state{
+                outbox = Outbox,
+                dir = Dir,
+                write_interval = WriteInterval,
+                read_interval = ReadInterval,
+                max_writes = MaxWrites
+            },
+            State1 = schedule_write(State0, 0),
+            State2 = schedule_read(State1, ReadInterval),
+            {ok, State2};
+        {error, Reason} ->
+            {stop, Reason}
+    end.
+
+-spec handle_call(term(), gen_server:from(), #state{}) ->
+    {reply, term(), #state{}}.
+handle_call(stats, _From, State) ->
+    {reply, state_stats(State), State};
+handle_call(_Request, _From, State) ->
+    {reply, {error, unknown_call}, State}.
+
+-spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
+handle_cast(_Request, State) ->
+    {noreply, State}.
+
+-spec handle_info(term(), #state{}) -> {noreply, #state{}}.
+handle_info(write_tick, State0 = #state{write_timer = WriteTimer}) ->
+    State1 = State0#state{write_timer = undefined},
+    State2 = case can_write(State1) of
+        true ->
+            append_one(State1);
+        false ->
+            State1
+    end,
+    State3 = case can_write(State2) of
+        true ->
+            schedule_write(State2, State2#state.write_interval);
+        false ->
+            State2
+    end,
+    _ = WriteTimer,
+    {noreply, State3};
+handle_info(read_tick, State0 = #state{read_timer = ReadTimer}) ->
+    State1 = State0#state{read_timer = undefined},
+    State2 = read_one(State1),
+    State3 = schedule_read(State2, State2#state.read_interval),
+    _ = ReadTimer,
+    {noreply, State3};
+handle_info(_Info, State) ->
+    {noreply, State}.
+
+-spec terminate(term(), #state{}) -> ok.
+terminate(_Reason, #state{outbox = Outbox, write_timer = WriteTimer, read_timer = ReadTimer}) ->
+    cancel_timer(WriteTimer),
+    cancel_timer(ReadTimer),
+    efka_iot_outbox:close(Outbox),
+    ok.
+
+-spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
+code_change(_OldVsn, State, _Extra) ->
+    {ok, State}.
+
+-spec append_one(#state{}) -> #state{}.
+append_one(State = #state{outbox = Outbox, write_count = WriteCount}) ->
+    PacketId = WriteCount + 1,
+    Packet = term_to_binary(#{
+        source => efka_iot_outbox_test,
+        packet_id => PacketId,
+        monotonic_time => erlang:monotonic_time(millisecond)
+    }),
+    case efka_iot_outbox:append(Packet, Outbox) of
+        {ok, NOutbox} ->
+            State#state{
+                outbox = NOutbox,
+                write_count = PacketId,
+                appended_count = State#state.appended_count + 1
+            };
+        {dropped, capacity_reached, NOutbox} ->
+            logger:warning("[efka_iot_outbox_test] outbox capacity reached, packet_id: ~p", [PacketId]),
+            State#state{
+                outbox = NOutbox,
+                write_count = PacketId,
+                dropped_count = State#state.dropped_count + 1
+            };
+        {error, Reason} ->
+            logger:warning("[efka_iot_outbox_test] append failed, packet_id: ~p, reason: ~p", [PacketId, Reason]),
+            State#state{
+                write_count = PacketId,
+                error_count = State#state.error_count + 1
+            }
+    end.
+
+-spec read_one(#state{}) -> #state{}.
+read_one(State = #state{outbox = Outbox}) ->
+    case efka_iot_outbox:next(Outbox) of
+        {ok, Seq, Packet} ->
+            logger:debug("[efka_iot_outbox_test] read seq: ~p, packet: ~p", [Seq, decode_packet(Packet)]),
+            case efka_iot_outbox:ack(Seq, Outbox) of
+                {ok, NOutbox} ->
+                    State#state{
+                        outbox = NOutbox,
+                        read_count = State#state.read_count + 1
+                    };
+                {error, Reason} ->
+                    logger:warning("[efka_iot_outbox_test] ack failed, seq: ~p, reason: ~p", [Seq, Reason]),
+                    State#state{error_count = State#state.error_count + 1}
+            end;
+        eof ->
+            State;
+        {error, Reason} ->
+            logger:warning("[efka_iot_outbox_test] read failed, reason: ~p", [Reason]),
+            State#state{error_count = State#state.error_count + 1}
+    end.
+
+-spec schedule_write(#state{}, non_neg_integer()) -> #state{}.
+schedule_write(State, Delay) ->
+    Ref = erlang:send_after(Delay, self(), write_tick),
+    State#state{write_timer = Ref}.
+
+-spec schedule_read(#state{}, non_neg_integer()) -> #state{}.
+schedule_read(State, Delay) ->
+    Ref = erlang:send_after(Delay, self(), read_tick),
+    State#state{read_timer = Ref}.
+
+-spec cancel_timer(undefined | reference()) -> ok.
+cancel_timer(undefined) ->
+    ok;
+cancel_timer(Ref) ->
+    _ = erlang:cancel_timer(Ref),
+    ok.
+
+-spec can_write(#state{}) -> boolean().
+can_write(#state{max_writes = infinity}) ->
+    true;
+can_write(#state{write_count = WriteCount, max_writes = MaxWrites}) ->
+    WriteCount < MaxWrites.
+
+-spec state_stats(#state{}) -> map().
+state_stats(#state{
+    dir = Dir,
+    write_interval = WriteInterval,
+    read_interval = ReadInterval,
+    max_writes = MaxWrites,
+    write_count = WriteCount,
+    appended_count = AppendedCount,
+    read_count = ReadCount,
+    dropped_count = DroppedCount,
+    error_count = ErrorCount
+}) ->
+    #{
+        dir => Dir,
+        write_interval => WriteInterval,
+        read_interval => ReadInterval,
+        max_writes => MaxWrites,
+        write_count => WriteCount,
+        appended_count => AppendedCount,
+        read_count => ReadCount,
+        dropped_count => DroppedCount,
+        error_count => ErrorCount,
+        pending_count => AppendedCount - ReadCount
+    }.
+
+-spec normalize_options([option()] | map()) -> map().
+normalize_options(Options) when is_map(Options) ->
+    Options;
+normalize_options(Options) when is_list(Options) ->
+    maps:from_list(Options).
+
+-spec positive_option(atom(), map(), pos_integer()) -> pos_integer().
+positive_option(Key, Options, Default) ->
+    case maps:get(Key, Options, Default) of
+        Value when is_integer(Value), Value > 0 ->
+            Value;
+        Value ->
+            error({invalid_option, Key, Value})
+    end.
+
+-spec max_writes_option(map()) -> pos_integer() | infinity.
+max_writes_option(Options) ->
+    case maps:get(max_writes, Options, infinity) of
+        infinity ->
+            infinity;
+        Value when is_integer(Value), Value > 0 ->
+            Value;
+        Value ->
+            error({invalid_option, max_writes, Value})
+    end.
+
+-spec decode_packet(binary()) -> term().
+decode_packet(Packet) ->
+    try binary_to_term(Packet, [safe]) of
+        Term ->
+            Term
+    catch
+        error:_ ->
+            Packet
+    end.

config/sys.config

@@ -1,60 +1,60 @@
 [
   {efka, [
-    {root_dir, "/usr/local/code/efka"},
+    {dets_dir, "/var/lib/efka/dets/"},
 
-    {dets_dir, "/tmp/db/"},
-
-    {tcp_server, [
-      {port, 18088}
+    {websocket_server, [
+      {port, 18080},
+      {acceptors, 10},
+      {max_connections, 1024},
+      {backlog, 256}
     ]},
 
-    {tls_server, [
+    {iot_server, [
       {host, "localhost"},
-      {port, 443}
+      {tls_port, 1443},
+      {udp_port, 18080}
+    ]},
+
+    {heartbeat, [
+      {interval, 5000}
     ]},
 
     {auth, [
       {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"},
-      {username, "test"},
-      {salt, "salt2345"},
-      {token, "token124"}
+      {token, "zpxlkvmqwnbghytrujsdieofazxcvbnm"}
     ]}
 
   ]},
 
-  %% 系统日志配置，系统日志为lager, 支持日志按日期自动分割
-  {lager, [
-    {colored, true},
-    %% Whether to write a crash log, and where. Undefined means no crash logger.
-    {crash_log, "trade_hub.crash.log"},
-    %% Maximum size in bytes of events in the crash log - defaults to 65536
-    {crash_log_msg_size, 65536},
-    %% Maximum size of the crash log in bytes, before its rotated, set
-    %% to 0 to disable rotation - default is 0
-    {crash_log_size, 10485760},
-    %% What time to rotate the crash log - default is no time
-    %% rotation. See the README for a description of this format.
-    {crash_log_date, "$D0"},
-    %% Number of rotated crash logs to keep, 0 means keep only the
-    %% current one - default is 0
-    {crash_log_count, 5},
-    %% Whether to redirect error_logger messages into lager - defaults to true
-    {error_logger_redirect, true},
+  {docker, [
+    {root_dir, "/var/lib/efka/docker/"}
+  ]},
 
-    %% How big the gen_event mailbox can get before it is switched into sync mode
-    {async_threshold, 20},
-    %% Switch back to async mode, when gen_event mailbox size decrease from `async_threshold'
-    %% to async_threshold - async_threshold_window
-    {async_threshold_window, 5},
+  %% 系统日志配置，使用 OTP logger
+  {kernel, [
+    %% 设置 Logger 的 primary log level
+    {logger_level, debug},
+    {logger, [
+      {handler, default, logger_std_h,
+        #{
+          level => debug,
+          formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}}
+        }
+      },
+
+      {handler, disk, logger_disk_log_h,
+        #{
+          level => debug,
+          config => #{
+          file => "log/debug.log",
+          max_no_files => 10,
+          max_no_bytes => 524288000
+        },
+          formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}}
+        }
+      }
 
-    {handlers, [
-      %% debug | info | warning | error, 日志级别
-      {lager_console_backend, debug},
-      {lager_file_backend, [{file, "debug.log"}, {level, debug}, {size, 314572800}]},
-      {lager_file_backend, [{file, "notice.log"}, {level, notice}, {size, 314572800}]},
-      {lager_file_backend, [{file, "error.log"}, {level, error}, {size, 314572800}]},
-      {lager_file_backend, [{file, "info.log"}, {level, info}, {size, 314572800}]}
     ]}
-
   ]}
-].
+
+].

config/vm.args

@@ -3,7 +3,7 @@
 -setcookie efka_cookie
 -kernel start_group false
 
--mnesia dir '"/usr/local/var/mnesia/efka"'
+-mnesia dir '"/var/lib/efka/mnesia"'
 -mnesia dump_log_write_threshold 5000
 -mnesia dc_dump_limit 40
 

docs/container_command_docker_json.md

@@ -0,0 +1,803 @@
+# IOT 容器命令到 Docker JSON 的转换说明
+
+本文档描述 `efka` 收到 `iot` 下发的容器管理 command 后，接受的 Erlang map 格式，以及 deploy 命令中 `create` 参数如何转换成 Docker Engine API 接收的 JSON。
+
+对应代码：
+
+- command 接收入口：[src/iot/efka_iot_client.erl](/usr/local/code/cloudkit/efka/apps/efka/src/iot/efka_iot_client.erl:177)
+- 部署任务管理：[docker_deploy_manager.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deploy_manager.erl:36)
+- 部署执行：[docker_deployer.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deployer.erl:39)
+- Docker JSON 构造：[docker_container_builder.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_container_builder.erl:14)
+- Docker API 调用：[docker_commands.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_commands.erl:36)
+
+## 1. 协议入口
+
+`iot` 通过 TLS 长连接向 `efka` 下发容器命令：
+
+```erlang
+{<<"command">>, Ref, {<<"container">>, CommandMap}}
+```
+
+`efka` 执行后回复：
+
+```erlang
+{<<"command_response">>, Ref, {<<"container">>, Reply}}
+```
+
+`Reply` 取值：
+
+```erlang
+<<"ok">>
+{<<"ok">>, Result}
+{<<"error">>, Reason}
+```
+
+`Ref` 是 `crypto:strong_rand_bytes(16)` 生成的 16 字节 binary。网络帧只使用 `binary_to_term(PacketBin, [safe])` 可解码的 safe term；协议 label、业务 label、map key 和 action 都使用 binary。
+
+只有 `efka_iot_client` 处于 `activated` 状态时，容器命令才会正常执行；处于非 activated 状态时会返回错误。
+
+## 2. 容器命令 map
+
+`efka_iot_client` 收到网络协议里的 binary-key `CommandMap` 后，直接按 binary key 和 binary action 做函数参数匹配，不再做整包 atom-key 转换。
+
+- command 顶层、`target`、deploy `params`、`create` 中间结构都使用 binary key。
+- `<<"action">>` 的取值使用 binary：`<<"list">>`、`<<"deploy">>`、`<<"start">>`、`<<"stop">>`、`<<"kill">>`、`<<"remove">>`、`<<"config">>`。
+- `undefined` 作为 Erlang 已有 atom 可以直接出现在 safe term 中；协议不再使用额外的哨兵 binary 表示缺省值。
+
+下面各小节描述的是 `efka` 直接接收和处理的 map 格式。
+
+### list
+
+```erlang
+#{<<"action">> => <<"list">>}
+```
+
+执行：
+
+```erlang
+docker_commands:get_containers()
+```
+
+Docker API：
+
+```http
+GET /containers/json?all=true
+```
+
+### deploy
+
+```erlang
+#{
+    <<"action">> => <<"deploy">>,
+    <<"task_id">> => TaskId,
+    <<"params">> => Params
+}
+```
+
+执行：
+
+```erlang
+docker_deploy_manager:deploy(TaskId, Params)
+```
+
+`deploy/2` 会启动独立部署进程，HTTP command response 只表示部署任务是否成功启动。实际部署过程和结果通过 `task_event` 消息流上报给 `iot`。
+
+### start
+
+```erlang
+#{
+    <<"action">> => <<"start">>,
+    <<"target">> => Target
+}
+```
+
+执行：
+
+```erlang
+docker_commands:start_container(ContainerNameOrId)
+```
+
+Docker API：
+
+```http
+POST /containers/{name_or_id}/start
+```
+
+### stop
+
+```erlang
+#{
+    <<"action">> => <<"stop">>,
+    <<"target">> => Target,
+    <<"timeout_seconds">> => TimeoutSeconds
+}
+```
+
+执行：
+
+```erlang
+docker_commands:stop_container(ContainerNameOrId, TimeoutSeconds)
+```
+
+Docker API：
+
+```http
+POST /containers/{name_or_id}/stop?t={TimeoutSeconds}
+```
+
+### kill
+
+```erlang
+#{
+    <<"action">> => <<"kill">>,
+    <<"target">> => Target,
+    <<"signal">> => Signal
+}
+```
+
+执行：
+
+```erlang
+docker_commands:kill_container(ContainerNameOrId, Signal)
+```
+
+### remove
+
+```erlang
+#{
+    <<"action">> => <<"remove">>,
+    <<"target">> => Target,
+    <<"force">> => Force,
+    <<"remove_volumes">> => RemoveVolumes
+}
+```
+
+执行：
+
+```erlang
+docker_commands:remove_container(ContainerNameOrId, Force, RemoveVolumes)
+```
+
+### config
+
+```erlang
+#{
+    <<"action">> => <<"config">>,
+    <<"target">> => Target,
+    <<"config">> => Config
+}
+```
+
+执行：
+
+```erlang
+docker_helper:update_container_config(ContainerNameOrId, iolist_to_binary(Config))
+```
+
+该命令不会调用 Docker API，只会更新 `efka` 主机上对应容器目录里的 `service.conf`。
+
+## 3. Target 解析规则
+
+`target` 是 map：
+
+```erlang
+#{
+    <<"name">> => ContainerName,
+    <<"id">> => ContainerId
+}
+```
+
+解析规则：
+
+- 优先使用 `name`。
+- `name` 为空时使用 `id`。
+- `name` 和 `id` 都为空时会触发匹配错误。
+
+## 4. deploy params 格式
+
+`deploy` 的 `params` 必须包含：
+
+```erlang
+#{
+    <<"container_name">> => ContainerName,
+    <<"create">> => Create
+}
+```
+
+字段说明：
+
+| 字段 | 类型 | 说明 |
+| --- | --- | --- |
+| `container_name` | binary | 容器名称。用于容器创建 URL 的 `name` 参数，也会注入环境变量 `CONTAINER_NAME`。 |
+| `create` | map | Docker create options 的中间结构，由 `iot` 侧构造，`efka` 侧补丁后转为 Docker JSON。 |
+
+`docker_deploy_manager` 会根据 `container_name` 按系统默认规则确保容器目录存在。目录固定为：
+
+```text
+docker.root_dir/container_name/
+```
+
+即使旧客户端在 `params` 中携带 `container_dir`，当前 deploy 流程也会忽略它，避免 HTTP 调用方控制 efka 主机上的写入目录。
+
+## 5. create 中间结构
+
+`create` 的结构：
+
+```erlang
+#{
+    <<"config">> => ContainerConfig,
+    <<"host_config">> => HostConfig,
+    <<"networking_config">> => NetworkingConfig
+}
+```
+
+这三个 map 会被 `docker_container_builder:build_options/3` 转成 Docker Engine API JSON。
+
+## 6. efka 自动补丁规则
+
+在转 Docker JSON 前，`efka` 会先执行补丁：
+
+### 环境变量
+
+向 `create.config.env` 前置注入：
+
+```erlang
+<<"CONTAINER_NAME=", ContainerName/binary>>
+```
+
+如果原 env 列表已经包含完全相同的值，则不会重复添加。
+
+### 配置文件 volume
+
+向 `create.config.volumes` 前置注入容器内配置路径：
+
+```erlang
+<<"/usr/local/etc/service.conf">>
+```
+
+### 配置文件 bind
+
+向 `create.host_config.binds` 前置注入宿主机配置文件映射：
+
+```erlang
+<<ConfigFile/binary, ":/usr/local/etc/service.conf">>
+```
+
+其中 `ConfigFile` 是当前容器目录下的：
+
+```text
+service.conf
+```
+
+如果列表里已存在完全相同的 bind，则不会重复添加。
+
+## 7. Docker JSON 顶层结构
+
+`docker_commands:create_container/2` 最终调用：
+
+```http
+POST /containers/create?name={container_name}
+Content-Type: application/json
+```
+
+请求 body 来自：
+
+```erlang
+Options = docker_container_builder:build_options(ContainerName, ContainerDir, Create),
+Body = iolist_to_binary(json:encode(Options))
+```
+
+最终 JSON 顶层字段：
+
+```json
+{
+  "Image": "...",
+  "Cmd": [],
+  "Entrypoint": [],
+  "Env": [],
+  "Labels": {},
+  "Volumes": {},
+  "User": "",
+  "WorkingDir": "",
+  "Hostname": "",
+  "ExposedPorts": {},
+  "NetworkingConfig": {},
+  "Healthcheck": {},
+  "HostConfig": {}
+}
+```
+
+空列表或空 map 字段有的会保留，有的会被省略到 `{}`，具体见下面映射规则。
+
+## 8. create.config 到 Docker JSON
+
+| 中间字段 | Docker JSON 字段 | 转换规则 |
+| --- | --- | --- |
+| `image` | `Image` | 转成 binary；缺省为 `""`。部署流程拉镜像时会对 image 补 `:latest`，但 create JSON 使用传入值。 |
+| `cmd` | `Cmd` | 列表元素逐个转 binary。 |
+| `entrypoint` | `Entrypoint` | 列表元素逐个转 binary。 |
+| `env` | `Env` | 列表元素逐个转 binary，并自动注入 `CONTAINER_NAME=...`。 |
+| `labels` | `Labels` | key/value 都转 binary。 |
+| `volumes` | `Volumes` | 转成 Docker 要求的 object，key 是容器内路径，value 是 `{}`。 |
+| `user` | `User` | 转成 binary；缺省为 `""`。 |
+| `working_dir` | `WorkingDir` | 转成 binary；缺省为 `""`。 |
+| `hostname` | `Hostname` | 转成 binary；缺省为 `""`。 |
+| `exposed_ports` | `ExposedPorts` | 转成 Docker 端口 object。 |
+| `healthcheck` | `Healthcheck` | 转成 Docker Healthcheck object；未传时为 `{}`。 |
+
+### Volumes
+
+输入：
+
+```erlang
+[<<"/data">>, <<"/usr/local/etc/service.conf">>]
+```
+
+输出 JSON：
+
+```json
+{
+  "Volumes": {
+    "/data": {},
+    "/usr/local/etc/service.conf": {}
+  }
+}
+```
+
+### ExposedPorts
+
+输入：
+
+```erlang
+[
+    #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
+    #{<<"container_port">> => 53, <<"protocol">> => <<"udp">>}
+]
+```
+
+输出 JSON：
+
+```json
+{
+  "ExposedPorts": {
+    "80/tcp": {},
+    "53/udp": {}
+  }
+}
+```
+
+规则：
+
+- `protocol` 为空或 `<<"tcp">>` 时输出 `tcp`。
+- 其他 protocol 原样输出。
+
+### Healthcheck
+
+该字段由 iot 校验后下发。`test` 必须是 binary list，`interval_ns`、`timeout_ns` 和 `retries` 必须是非负整数；efka 只按收到的中间 map 转成 Docker JSON 字段。
+
+输入：
+
+```erlang
+#{
+    <<"test">> => [<<"CMD-SHELL">>, <<"curl -f http://localhost || exit 1">>],
+    <<"interval_ns">> => 30000000000,
+    <<"timeout_ns">> => 10000000000,
+    <<"retries">> => 3
+}
+```
+
+输出 JSON：
+
+```json
+{
+  "Healthcheck": {
+    "Test": ["CMD-SHELL", "curl -f http://localhost || exit 1"],
+    "Interval": 30000000000,
+    "Timeout": 10000000000,
+    "Retries": 3
+  }
+}
+```
+
+## 9. create.host_config 到 Docker JSON
+
+`host_config` 会被合并到 Docker JSON 的 `HostConfig` 字段。
+
+| 中间字段 | Docker JSON 字段 | 转换规则 |
+| --- | --- | --- |
+| `binds` | `HostConfig.Binds` | 列表元素逐个转 binary，并自动注入 `service.conf` bind。空列表时省略。 |
+| `network_mode` | `HostConfig.NetworkMode` | 空 binary 时省略。 |
+| `restart_policy` | `HostConfig.RestartPolicy` | 转成 `Name` 和可选 `MaximumRetryCount`。 |
+| `privileged` | `HostConfig.Privileged` | 只有 true 时输出。false 时省略。 |
+| `cap_add` / `cap_drop` | `HostConfig.CapAdd` / `HostConfig.CapDrop` | 两者都为空时省略；只要一个非空，两个字段都会输出。 |
+| `devices` | `HostConfig.Devices` | 转成 Docker device object 列表。 |
+| `memory` | `HostConfig.Memory` | 0 时省略。 |
+| `memory_reservation` | `HostConfig.MemoryReservation` | 0 时省略。 |
+| `nano_cpus` | `HostConfig.NanoCpus` | 0 时省略。 |
+| `cpu_shares` | `HostConfig.CpuShares` | 0 时省略。 |
+| `port_bindings` | `HostConfig.PortBindings` | 转成 Docker 端口绑定 object；空列表时省略。 |
+| `ulimits` | `HostConfig.Ulimits` | 空列表时省略。 |
+| `tmpfs` | `HostConfig.Tmpfs` | 空 map 时省略。 |
+| `sysctls` | `HostConfig.Sysctls` | 空 map 时省略。 |
+| `extra_hosts` | `HostConfig.ExtraHosts` | 空列表时省略。 |
+
+### Binds
+
+输入：
+
+```erlang
+[
+    <<"/host/data:/data">>,
+    <<"/host/log:/var/log:ro">>
+]
+```
+
+efka 自动补丁后输出：
+
+```json
+{
+  "HostConfig": {
+    "Binds": [
+      "/path/to/container/service.conf:/usr/local/etc/service.conf",
+      "/host/data:/data",
+      "/host/log:/var/log:ro"
+    ]
+  }
+}
+```
+
+### RestartPolicy
+
+输入：
+
+```erlang
+#{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0}
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "RestartPolicy": {
+      "Name": "always"
+    }
+  }
+}
+```
+
+输入：
+
+```erlang
+#{<<"name">> => <<"on-failure">>, <<"maximum_retry_count">> => 3}
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "RestartPolicy": {
+      "Name": "on-failure",
+      "MaximumRetryCount": 3
+    }
+  }
+}
+```
+
+### Devices
+
+输入：
+
+```erlang
+[
+    #{
+        <<"path_on_host">> => <<"/dev/ttyUSB0">>,
+        <<"path_in_container">> => <<"/dev/ttyUSB0">>,
+        <<"cgroup_permissions">> => <<"rwm">>
+    }
+]
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "Devices": [
+      {
+        "PathOnHost": "/dev/ttyUSB0",
+        "PathInContainer": "/dev/ttyUSB0",
+        "CgroupPermissions": "rwm"
+      }
+    ]
+  }
+}
+```
+
+`cgroup_permissions` 为空时默认输出 `rwm`。
+
+### PortBindings
+
+输入：
+
+```erlang
+[
+    #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
+    #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
+]
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "PortBindings": {
+      "80/tcp": [
+        {"HostIp": "", "HostPort": "8080"}
+      ],
+      "443/tcp": [
+        {"HostIp": "", "HostPort": "443"}
+      ]
+    }
+  }
+}
+```
+
+规则：
+
+- key 使用容器端口和协议组成：`container_port/protocol`。
+- `protocol` 为空或 `tcp` 时输出 `tcp`。
+- `HostPort` 按 Docker API 要求输出为字符串。
+
+### 资源限制
+
+输入：
+
+```erlang
+#{
+    <<"memory">> => 536870912,
+    <<"memory_reservation">> => 268435456,
+    <<"nano_cpus">> => 1500000000,
+    <<"cpu_shares">> => 512
+}
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "Memory": 536870912,
+    "MemoryReservation": 268435456,
+    "NanoCpus": 1500000000,
+    "CpuShares": 512
+  }
+}
+```
+
+值为 0 的资源字段会被省略。
+
+### Ulimits
+
+输入：
+
+```erlang
+[
+    #{<<"name">> => <<"nofile">>, <<"soft">> => 1024, <<"hard">> => 2048}
+]
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "Ulimits": [
+      {
+        "Name": "nofile",
+        "Soft": 1024,
+        "Hard": 2048
+      }
+    ]
+  }
+}
+```
+
+### Tmpfs
+
+输入：
+
+```erlang
+#{
+    <<"/tmp">> => <<>>,
+    <<"/run">> => <<"rw,size=64m">>
+}
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "Tmpfs": {
+      "/tmp": "",
+      "/run": "rw,size=64m"
+    }
+  }
+}
+```
+
+### Sysctls
+
+输入：
+
+```erlang
+#{<<"net.ipv4.ip_forward">> => <<"1">>}
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "Sysctls": {
+      "net.ipv4.ip_forward": "1"
+    }
+  }
+}
+```
+
+### ExtraHosts
+
+输入：
+
+```erlang
+[<<"host.docker.internal:host-gateway">>]
+```
+
+输出：
+
+```json
+{
+  "HostConfig": {
+    "ExtraHosts": ["host.docker.internal:host-gateway"]
+  }
+}
+```
+
+## 10. create.networking_config 到 Docker JSON
+
+输入：
+
+```erlang
+#{
+    <<"endpoints">> => [
+        #{<<"name">> => <<"bridge">>},
+        #{<<"name">> => <<"mynet">>}
+    ]
+}
+```
+
+输出：
+
+```json
+{
+  "NetworkingConfig": {
+    "EndpointsConfig": {
+      "bridge": {},
+      "mynet": {}
+    }
+  }
+}
+```
+
+未传或 endpoints 为空时输出：
+
+```json
+{
+  "NetworkingConfig": {}
+}
+```
+
+## 11. 完整转换示例
+
+收到的 deploy command：
+
+```erlang
+{<<"command">>, Ref, {<<"container">>, #{
+    <<"action">> => <<"deploy">>,
+    <<"task_id">> => 1001,
+    <<"params">> => #{
+        <<"container_name">> => <<"my_nginx">>,
+        <<"create">> => #{
+            <<"config">> => #{
+                <<"image">> => <<"docker.io/library/nginx:latest">>,
+                <<"cmd">> => [<<"nginx">>, <<"-g">>, <<"daemon off;">>],
+                <<"env">> => [<<"ENV=prod">>],
+                <<"volumes">> => [<<"/data">>],
+                <<"exposed_ports">> => [#{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>}]
+            },
+            <<"host_config">> => #{
+                <<"binds">> => [<<"/host/data:/data">>],
+                <<"restart_policy">> => #{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0},
+                <<"memory">> => 536870912,
+                <<"port_bindings">> => [
+                    #{
+                        <<"host_ip">> => <<>>,
+                        <<"host_port">> => 8080,
+                        <<"container_port">> => 80,
+                        <<"protocol">> => <<"tcp">>
+                    }
+                ]
+            },
+            <<"networking_config">> => #{
+                <<"endpoints">> => [#{<<"name">> => <<"bridge">>}]
+            }
+        }
+    }
+}}}
+```
+
+生成的 Docker JSON 示意：
+
+```json
+{
+  "Image": "docker.io/library/nginx:latest",
+  "Cmd": ["nginx", "-g", "daemon off;"],
+  "Entrypoint": [],
+  "Env": ["CONTAINER_NAME=my_nginx", "ENV=prod"],
+  "Labels": {},
+  "Volumes": {
+    "/usr/local/etc/service.conf": {},
+    "/data": {}
+  },
+  "User": "",
+  "WorkingDir": "",
+  "Hostname": "",
+  "ExposedPorts": {
+    "80/tcp": {}
+  },
+  "NetworkingConfig": {
+    "EndpointsConfig": {
+      "bridge": {}
+    }
+  },
+  "Healthcheck": {},
+  "HostConfig": {
+    "Binds": [
+      "/efka/root/my_nginx/service.conf:/usr/local/etc/service.conf",
+      "/host/data:/data"
+    ],
+    "RestartPolicy": {
+      "Name": "always"
+    },
+    "Memory": 536870912,
+    "PortBindings": {
+      "80/tcp": [
+        {
+          "HostIp": "",
+          "HostPort": "8080"
+        }
+      ]
+    }
+  }
+}
+```
+
+## 12. 部署流程补充
+
+deploy action 的执行流程：
+
+1. 根据 `container_name` 和 `docker.root_dir` 确保默认容器目录存在。
+2. 启动独立部署进程。
+3. 部署进程上报任务事件流。
+4. 规范化镜像名：如果镜像最后一段没有 tag，则补 `:latest`。
+5. 调用 Docker API 拉取镜像。
+6. 构造 Docker create JSON。
+7. 调用 `POST /containers/create?name={container_name}`。
+8. 创建空 `service.conf` 文件。
+9. 写入部署摘要日志并关闭任务事件流。
+
+注意：当前 `ensure_container_absent/2` 只上报“开始创建容器”，不会删除已有同名容器；如果 Docker 返回名称冲突，部署会失败并上报“本地容器已经存在”。

docs/efka_iot_protocol.md

@@ -0,0 +1,274 @@
+# EFKA 与 IOT 交互协议
+
+本文档描述 `efka` 与 `iot` 之间的 TLS 长连接协议。当前协议由 Erlang term 直接序列化，发送端使用 `term_to_binary/1`，接收端使用 `binary_to_term(PacketBin, [safe])`。
+
+协议帧只使用 safe external term：顶层 label、业务 label、map key 使用 binary；`Ref` 使用 `crypto:strong_rand_bytes(16)` 生成，是 16 字节 binary。网络协议里不发送 Erlang `reference()`，也不依赖动态创建 atom。
+
+## 传输层
+
+- `efka` 作为 TLS client 连接 `iot`。
+- `iot` 作为 TLS server 接收多个 `efka` 连接，一个连接对应一个 `ssl_channel` 进程。
+- socket 使用 `{packet, 4}`，每个 Erlang term binary 作为一个完整包发送。
+- `Ref` 使用 `crypto:strong_rand_bytes(16)` 生成，只在当前连接的 inflight 表内匹配。
+
+## 顶层帧
+
+协议顶层 tuple 用来表达交互语义：
+
+```erlang
+{<<"request">>, Ref, Body}
+{<<"response">>, Ref, Reply}
+{<<"command">>, Ref, {Domain, Payload}}
+{<<"command_response">>, Ref, {Domain, Reply}}
+{<<"message">>, Body}
+```
+
+语义说明：
+
+| 帧 | 方向 | 语义 |
+| --- | --- | --- |
+| `{<<"request">>, Ref, Body}` | efka -> iot | efka 发起请求，需要 iot 回复 |
+| `{<<"response">>, Ref, Reply}` | iot -> efka | iot 对 efka request 的回复 |
+| `{<<"command">>, Ref, {Domain, Payload}}` | iot -> efka | iot 下发命令，需要 efka 回复 |
+| `{<<"command_response">>, Ref, {Domain, Reply}}` | efka -> iot | efka 对 iot command 的回复 |
+| `{<<"message">>, Body}` | 双向 | 异步消息，不要求回复 |
+
+`command` 和 `command_response` 的 `Domain` 表示业务域，目前支持：
+
+- `<<"container">>`
+
+## 鉴权请求
+
+初始连接由 `efka` 发起鉴权 request。每条 TLS 连接只允许一次鉴权；`iot` 侧鉴权成功后会在 `ssl_channel` 标记该连接已鉴权，如果同一连接再次发送 `auth_request`，`iot` 会直接关闭连接。
+
+```erlang
+{<<"request">>, Ref, {<<"auth_request">>, #{
+    <<"uuid">> => UUID,
+    <<"token">> => Token,
+    <<"timestamp">> => Timestamp
+}}}
+```
+
+`iot` 回复：
+
+```erlang
+{<<"response">>, Ref, {<<"auth_response">>, <<"ok">>}}
+{<<"response">>, Ref, {<<"auth_response">>, {<<"error">>, {<<"failed">>, Reason}}}}
+```
+
+处理语义：
+
+- `<<"ok">>`：`efka` 进入 `activated` 状态。
+- `{<<"error">>, {<<"failed">>, Reason}}`：鉴权失败，`iot` 返回失败响应后关闭连接；`efka` 进入重连流程。
+
+## 授权控制
+
+`/host/activate` 只修改 `iot` 本地和持久化的 host 授权状态，不再向 `efka` 下发 auth command。`efka` 可以继续保持连接并发送数据，是否处理这些数据由 `iot_host` 当前状态决定。
+
+因此当前协议没有 `{<<"command">>, Ref, {<<"auth">>, ...}}` 和 `{<<"command_response">>, Ref, {<<"auth">>, ...}}`。授权关闭时，`iot_host` 保持 channel 在线，但不处理上报数据；授权重新打开后，已在线的 channel 可以继续使用。
+
+## 容器管理命令
+
+`iot` 对 `efka` 的容器管理使用 command 语义：
+
+```erlang
+{<<"command">>, Ref, {<<"container">>, CommandMap}}
+```
+
+`efka` 回复：
+
+```erlang
+{<<"command_response">>, Ref, {<<"container">>, Reply}}
+```
+
+`Reply` 取值：
+
+```erlang
+<<"ok">>
+{<<"ok">>, Result}
+{<<"error">>, Reason}
+```
+
+`CommandMap` 使用 binary key 和 binary action；`efka` 接收后直接按 binary key/action 匹配，Docker 参数链路继续使用 binary-key map，不再转换成 atom-key map。
+
+### list
+
+```erlang
+#{<<"action">> => <<"list">>}
+```
+
+返回当前 `efka` 主机上的容器列表。
+
+### deploy
+
+```erlang
+#{
+    <<"action">> => <<"deploy">>,
+    <<"task_id">> => TaskId,
+    <<"params">> => Params
+}
+```
+
+触发容器部署。部署过程中的流式日志不通过该 command response 返回，而是通过 `message` 的 `task_event` 上报。
+
+### start
+
+```erlang
+#{
+    <<"action">> => <<"start">>,
+    <<"target">> => Target
+}
+```
+
+### stop
+
+```erlang
+#{
+    <<"action">> => <<"stop">>,
+    <<"target">> => Target,
+    <<"timeout_seconds">> => TimeoutSeconds
+}
+```
+
+### kill
+
+```erlang
+#{
+    <<"action">> => <<"kill">>,
+    <<"target">> => Target,
+    <<"signal">> => Signal
+}
+```
+
+### remove
+
+```erlang
+#{
+    <<"action">> => <<"remove">>,
+    <<"target">> => Target,
+    <<"force">> => Force,
+    <<"remove_volumes">> => RemoveVolumes
+}
+```
+
+### config
+
+```erlang
+#{
+    <<"action">> => <<"config">>,
+    <<"target">> => Target,
+    <<"config">> => Config
+}
+```
+
+更新容器配置文件。
+
+### Target
+
+容器目标使用 map 表示：
+
+```erlang
+#{
+    <<"name">> => ContainerName,
+    <<"id">> => ContainerId
+}
+```
+
+`name` 和 `id` 至少一个非空；优先使用 `name`，`name` 为空时使用 `id`。
+
+## 异步消息
+
+`message` 不带 `Ref`，不要求对端回复。
+
+### efka -> iot: data
+
+```erlang
+{<<"message">>, {<<"data">>, #{
+    <<"route_key">> => RouteKey,
+    <<"metric">> => Metric
+}}}
+```
+
+用于 `efka` 上报业务指标数据。
+
+### efka -> iot: task_event
+
+```erlang
+{<<"message">>, {<<"task_event">>, #{
+    <<"task_id">> => TaskId,
+    <<"type">> => Type,
+    <<"stream">> => Stream
+}}}
+```
+
+任务事件流关闭时：
+
+```erlang
+{<<"message">>, {<<"task_event">>, #{
+    <<"task_id">> => TaskId,
+    <<"type">> => <<"close">>,
+    <<"stream">> => Reason
+}}}
+```
+
+`task_event` 本身不携带 `uuid`。`iot` 在接收该消息时使用当前已鉴权 `ssl_channel` 绑定的 host UUID，把事件路由到内部任务进程 `{UUID, TaskId}`。HTTP 页面通过 SSE 订阅时也必须使用同一组参数：
+
+```http
+GET /event_stream?uuid=<host_uuid>&task_id=<task_id>
+```
+
+`iot` 会为每个 `{UUID, TaskId}` 维护一个独立的任务进程，用于缓存最近的部署日志、支持多个 SSE listener，并在收到 close 事件后结束事件流。
+
+### efka -> iot: ping
+
+```erlang
+{<<"message">>, <<"ping">>}
+```
+
+用于 TLS 长连接的应用层保活。`efka` 在鉴权成功后周期发送，当前发送间隔为 30 秒。
+
+`iot` 收到后回复：
+
+```erlang
+{<<"message">>, <<"pong">>}
+```
+
+`ping/pong` 只表示 TLS 连接仍可读写，不参与 host online/offline 判定。host 上下线仍由 UDP 心跳和 `iot_host` 本地连接状态共同维护。
+
+### iot -> efka: pub
+
+```erlang
+{<<"message">>, {<<"pub">>, #{
+    <<"topic">> => Topic,
+    <<"qos">> => Qos,
+    <<"content">> => Content
+}}}
+```
+
+用于 `iot` 向 `efka` 本地订阅系统发布 topic 消息。
+
+## 状态与超时
+
+- `efka` 鉴权超时时间：5 秒。
+- `iot` command inflight 超时时间：60 秒。
+- `iot` SSL channel 空闲超时时间：120 秒。120 秒内没有收到任何 TLS 包，包括 `ping`、业务 `message`、`request`、`command_response`，`iot` 会主动关闭该连接。
+- `iot` 管理多个 `efka` 时，每个连接有独立 `ssl_channel` 和独立 inflight 表。
+- command 超时后，`iot` 删除 inflight 记录；之后如果迟到的 `command_response` 到达，会被视为未预期响应。
+
+## UDP 心跳
+
+`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。TLS control channel 和 UDP 心跳共用 `iot_server.host`，分别使用 `tls_port` 和 `udp_port`。UDP 心跳包使用 HMAC-SHA256 校验，HMAC key 为 `SHA256(auth.token)`。
+
+详细格式见 [heartbeat.md](heartbeat.md)。
+
+## 兼容性
+
+当前协议不兼容旧 tuple：
+
+- 旧容器管理：`{request, Ref, {container_request, ...}}`
+- 旧容器回复：`{response, Ref, {container_response, ...}}`
+- 旧授权控制：`{message, {auth_control, Command}}`
+- 已移除的 auth command：`{command, Ref, {auth, activate | deactivate}}`
+- 旧 Ref：Erlang `reference()`，例如 `make_ref()` 生成的值。
+
+如果需要滚动升级，应先增加临时兼容分支或引入协议版本协商。

docs/heartbeat.md

@@ -0,0 +1,52 @@
+# UDP 心跳
+
+`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。该心跳只表示主机存活，不依赖 TLS control channel 是否在线。
+
+## 配置
+
+TLS 和 UDP 共用同一个 `iot_server.host`：
+
+```erlang
+{iot_server, [
+  {host, "localhost"},
+  {tls_port, 443},
+  {udp_port, 18080}
+]},
+
+{heartbeat, [
+  {interval, 5000}
+]},
+
+{auth, [
+  {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"},
+  {token, "zpxlkvmqwnbghytrujsdieofazxcvbnm"}
+]}
+```
+
+- `tls_port` 用于 `efka_iot_client` 建立 TLS 连接。
+- `udp_port` 用于 `efka_iot_heartbeat` 发送 UDP 心跳。
+- `heartbeat.interval` 是发送间隔，单位毫秒，默认 5000。
+- UDP 心跳和 TLS 鉴权复用 `auth.uuid` 和 `auth.token`。
+
+## 包格式
+
+```erlang
+<<
+    Version:8,
+    UuidLen:16,
+    UUID:UuidLen/binary,
+    Timestamp:64/unsigned-big,
+    Nonce:16/binary,
+    Mac:32/binary
+>>
+```
+
+`Mac` 使用 HMAC-SHA256：
+
+```erlang
+HeartbeatSecret = crypto:hash(sha256, Token),
+Payload = <<Version:8, UuidLen:16, UUID:UuidLen/binary, Timestamp:64/unsigned-big, Nonce:16/binary>>,
+Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload)
+```
+
+其中 `Token` 是 `auth.token`。`iot` 侧注册 efka client 时保存同样的 `SHA256(Token)` 派生值，并用它校验心跳。

docs/service_websocket_protocol.md

@@ -0,0 +1,439 @@
+# EFKA 与 Service WebSocket 交互协议
+
+本文档描述部署在 `efka` 主机上的 service 与 `efka` 之间的 WebSocket 通讯逻辑。当前接入实现位于：
+
+- [efka_service_channel.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service_channel.erl)
+- [service.proto](/usr/local/code/cloudkit/efka/apps/efka/proto/service.proto)
+- [efka_subscription.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_subscription.erl)
+- [efka_service.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service.erl)
+
+## 1. 连接入口
+
+`efka` 启动时会创建 Cowboy WebSocket server，监听配置来自 `efka.config` 的 `websocket_server`：
+
+```erlang
+{websocket_server, [
+    {port, 18080},
+    {acceptors, 10},
+    {max_connections, 1024},
+    {backlog, 1024}
+]}
+```
+
+WebSocket 路由固定为：
+
+```http
+GET /ws
+```
+
+连接建立后，每个 WebSocket 连接对应一个 `efka_service_channel` 进程。
+
+当前协议没有独立的认证字段。service 必须先发送 `register` request 完成注册；未注册前只能处理 `register`，其它 request 会返回 `invalid request`，cast 会被忽略。
+
+Cowboy WebSocket ping 帧由 `efka_service_channel` 直接回复 pong：
+
+```erlang
+websocket_handle(ping, State) -> {reply, pong, State}
+```
+
+## 2. 二进制帧格式
+
+service 与 `efka` 之间使用 WebSocket binary frame。每个业务 frame 的第 1 个字节是自定义帧类型，剩余字节是 protobuf payload。
+
+```text
+<frame_type:1 byte><protobuf_payload:N bytes>
+```
+
+帧类型：
+
+| frame_type | 十六进制 | 方向 | 语义 | protobuf 类型 |
+| --- | --- | --- | --- | --- |
+| `REQUEST` | `0x01` | service -> efka | 请求，需要 efka 回复 | `ServiceRequest` |
+| `REPLY` | `0x02` | efka -> service | request 的响应 | `ServiceReply` |
+| `CAST` | `0x03` | 双向 | 单向消息，不要求回复 | `ServiceCast` |
+
+`efka_service_channel` 当前只处理 service 发来的 `REQUEST` 和 `CAST`。收到未知格式时只记录日志，不主动发送错误响应。
+
+## 3. Protobuf 定义
+
+协议定义来自 `apps/efka/proto/service.proto`：
+
+```protobuf
+syntax = "proto3";
+
+message ServiceRequest {
+  uint32 packet_id = 1;
+
+  message Register {
+    string service_id = 1;
+  }
+
+  message Subscribe {
+    string topic = 1;
+  }
+
+  oneof request {
+    Register register = 10;
+    Subscribe subscribe = 11;
+  }
+}
+
+message ServiceReply {
+  uint32 packet_id = 1;
+
+  message Error {
+    int32 code = 1;
+    string message = 2;
+  }
+
+  oneof reply {
+    bytes result = 10;
+    Error error = 11;
+  }
+}
+
+message ServiceCast {
+  message MetricData {
+    bytes route_key = 1;
+    bytes metric = 2;
+  }
+
+  message TopicEvent {
+    string topic = 1;
+    bytes content = 2;
+  }
+
+  oneof body {
+    TopicEvent topic_event = 10;
+    MetricData metric_data = 11;
+  }
+}
+```
+
+注意：
+
+- `ServiceRequest.packet_id` 由 service 生成，用于匹配 request 和 reply。
+- `ServiceReply.packet_id` 必须等于原 request 的 `packet_id`。
+- `ServiceReply.result` 当前成功值一般为 `<<"ok">>`。
+- `ServiceReply.error.code` 当前失败时统一使用 `-1`。
+- `MetricData.route_key` 和 `MetricData.metric` 是 bytes。
+- `TopicEvent.topic` 是 string，`TopicEvent.content` 是 bytes。
+
+## 4. request/reply 语义
+
+### 4.1 register
+
+service 建立 WebSocket 后应先发送注册请求：
+
+```protobuf
+ServiceRequest {
+  packet_id: 1
+  register {
+    service_id: "service-a"
+  }
+}
+```
+
+WebSocket binary frame：
+
+```text
+0x01 ++ protobuf(ServiceRequest)
+```
+
+`efka` 处理逻辑：
+
+1. `efka_service_channel` 解码 `ServiceRequest`。
+2. 调用 `efka_service_sup:start_service(ServiceId)` 启动或复用 service 进程。
+3. 调用 `efka_service:attach_channel(ServicePid, ChannelPid)` 绑定当前 WebSocket channel。
+4. 通过 `efka_service_model:insert/1` 写入 service 记录：
+
+```erlang
+#service{
+    service_id = ServiceId,
+    container_name = <<>>,
+    status = ?SERVICE_RUNNING,
+    meta_data = #{},
+    create_ts = efka_util:timestamp(),
+    update_ts = efka_util:timestamp()
+}
+```
+
+如果该 `service_id` 是新记录，`status` 会写入 `?SERVICE_RUNNING`。如果该 `service_id` 已存在，当前 `efka_service_model:insert/1` 只更新 `meta_data`、`container_name` 和 `update_ts`，不会覆盖已有 `status`。
+
+5. `efka_service_channel` 状态更新为：
+
+```erlang
+#state{
+    service_id = ServiceId,
+    service_pid = ServicePid,
+    is_registered = true
+}
+```
+
+成功响应：
+
+```protobuf
+ServiceReply {
+  packet_id: 1
+  result: "ok"
+}
+```
+
+WebSocket binary frame：
+
+```text
+0x02 ++ protobuf(ServiceReply)
+```
+
+失败响应：
+
+```protobuf
+ServiceReply {
+  packet_id: 1
+  error {
+    code: -1
+    message: "attach channel failed"
+  }
+}
+```
+
+常见失败场景：
+
+- 同一个 `service_id` 已有存活 channel，`efka_service:attach_channel/2` 返回 `{error, <<"channel exists">>}`。
+- 当前实现对外统一返回 `"attach channel failed"`，详细原因只写日志。
+
+### 4.2 subscribe
+
+注册成功后，service 可以订阅 topic：
+
+```protobuf
+ServiceRequest {
+  packet_id: 2
+  subscribe {
+    topic: "device/*/event"
+  }
+}
+```
+
+`efka` 处理逻辑：
+
+1. 要求当前 channel 已注册，即 `is_registered = true`。
+2. 调用 `efka_subscription:subscribe(Topic, ChannelPid)`。
+3. 订阅成功后，把 topic 加入 `efka_service_channel` 的 `subscribed_topics` 集合。
+
+成功响应：
+
+```protobuf
+ServiceReply {
+  packet_id: 2
+  result: "ok"
+}
+```
+
+失败响应：
+
+```protobuf
+ServiceReply {
+  packet_id: 2
+  error {
+    code: -1
+    message: "invalid topic name"
+  }
+}
+```
+
+未注册时发送 subscribe 会走通用错误：
+
+```protobuf
+ServiceReply {
+  packet_id: 2
+  error {
+    code: -1
+    message: "invalid request"
+  }
+}
+```
+
+## 5. cast 语义
+
+### 5.1 service -> efka: metric_data
+
+service 通过 `CAST` 上报指标数据：
+
+```protobuf
+ServiceCast {
+  metric_data {
+    route_key: "device/a/temp"
+    metric: "{\"value\":23.5}"
+  }
+}
+```
+
+WebSocket binary frame：
+
+```text
+0x03 ++ protobuf(ServiceCast)
+```
+
+`efka` 处理逻辑：
+
+1. `efka_service_channel` 解码 `ServiceCast`。
+2. 要求当前 channel 已注册。
+3. 调用：
+
+```erlang
+efka_service:metric_data(ServicePid, RouteKey, Metric)
+```
+
+4. `efka_service` 再调用：
+
+```erlang
+efka_iot_client:metric_data(RouteKey, Metric)
+```
+
+5. `efka_iot_client` 通过 efka 与 iot 的 TLS 长连接把数据上报给 iot：
+
+```erlang
+{<<"message">>, {<<"data">>, #{
+    <<"route_key">> => RouteKey,
+    <<"metric">> => Metric
+}}}
+```
+
+`metric_data` 是单向消息，service 不会收到 reply。未注册或未知 cast body 会被忽略。
+
+### 5.2 efka -> service: topic_event
+
+当 `iot` 通过 efka 与 iot 的 TLS channel 下发 pub 消息到 `efka` 时，`efka_iot_client` 会调用：
+
+```erlang
+efka_subscription:publish(Topic, Qos, Content)
+```
+
+`efka_subscription` 根据 topic 匹配已订阅的 `efka_service_channel`。匹配成功后向 channel 进程发送：
+
+```erlang
+{topic_broadcast, Topic, Content}
+```
+
+`efka_service_channel` 再推送 WebSocket `CAST` 给 service：
+
+```protobuf
+ServiceCast {
+  topic_event {
+    topic: "device/a/event"
+    content: "payload bytes"
+  }
+}
+```
+
+WebSocket binary frame：
+
+```text
+0x03 ++ protobuf(ServiceCast)
+```
+
+`topic_event` 是单向消息，不要求 service 回复。
+
+## 6. topic 匹配规则
+
+`efka_subscription` 当前使用 `/` 拆分 topic：
+
+```erlang
+binary:split(Topic, <<$/>>, [global])
+```
+
+支持两种通配符：
+
+| 通配符 | 语义 |
+| --- | --- |
+| `*` | 单级匹配，只匹配一个 segment |
+| `+` | 多级匹配，只能出现在最后一段，且至少匹配一个剩余 segment |
+
+示例：
+
+| 订阅 topic | 发布 topic | 是否匹配 |
+| --- | --- | --- |
+| `device/a/temp` | `device/a/temp` | 是 |
+| `device/*/temp` | `device/a/temp` | 是 |
+| `device/*/temp` | `device/a/b/temp` | 否 |
+| `device/+` | `device/a` | 是 |
+| `device/+` | `device/a/temp` | 是 |
+| `device/+` | `device` | 否 |
+
+同一个 `efka_service_channel` 订阅多个 topic 时，同一次 publish 最多投递一次。
+
+## 7. QoS 与遗留消息
+
+`efka_subscription:publish(Topic, Qos, Content)` 中的 `Qos` 影响没有订阅者时的行为：
+
+- `Qos = 0`：没有匹配订阅者时直接丢弃。
+- `Qos /= 0`：没有匹配订阅者时暂存到 `remaining_messages`。
+
+当后续有 service 订阅能匹配这些 topic 时，`efka_subscription` 会把对应遗留消息推送给该 service，并从 `remaining_messages` 移除。
+
+当前 `remaining_messages` 存在内存中，不持久化；`efka` 重启后会丢失。
+
+## 8. 生命周期与清理
+
+### WebSocket channel 关闭
+
+`efka_service_channel:terminate/3` 会执行：
+
+1. 调用 `efka_subscription:unsubscribe_all(ChannelPid)` 清理该 channel 的所有订阅。
+2. 如果 channel 已完成 register，则调用：
+
+```erlang
+efka_service_model:change_status(ServiceId, ?SERVICE_STOPPED)
+```
+
+也就是把 service 状态更新为停止。
+
+### service 进程退出
+
+`efka_service_channel` monitor 绑定的 `efka_service` 进程。如果 service 进程退出，channel 会停止：
+
+```erlang
+websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State) ->
+    {stop, State#state{service_pid = undefined}}
+```
+
+`efka_service` 同时也 monitor channel。channel 退出时，`efka_service` 会清空自身的 `channel_pid`。
+
+## 9. 完整交互流程
+
+典型流程：
+
+```text
+service                         efka/efka_service_channel              efka_service              efka_subscription
+  |                                      |                              |                            |
+  |-- WebSocket connect /ws ------------>|                              |                            |
+  |                                      |                              |                            |
+  |-- REQUEST register ----------------->|                              |                            |
+  |                                      |-- start_service(service_id) ->|                            |
+  |                                      |-- attach_channel ----------->|                            |
+  |<------------- REPLY result="ok" -----|                              |                            |
+  |                                      |                              |                            |
+  |-- REQUEST subscribe ---------------->|                              |                            |
+  |                                      |---------------- subscribe ------------------------------>|
+  |<------------- REPLY result="ok" -----|                              |                            |
+  |                                      |                              |                            |
+  |-- CAST metric_data ----------------->|                              |                            |
+  |                                      |-- metric_data -------------->|                            |
+  |                                      |                              |-- efka_iot_client:metric_data -> iot
+  |                                      |                              |                            |
+  |<------------- CAST topic_event ------|<--------------- topic_broadcast -----------------------|
+  |                                      |                              |                            |
+  |-- WebSocket close ------------------>|                              |                            |
+  |                                      |-- unsubscribe_all -------------------------------------->|
+  |                                      |-- change_status(service_id, stopped)
+```
+
+## 10. 当前限制
+
+- 没有认证字段，service 只通过 `service_id` 注册。
+- `register` 可重复连接同一个 `service_id`，但同一时间只允许一个活跃 channel 绑定到 `efka_service`。
+- request 只支持 `register` 和 `subscribe`。
+- cast 只支持 `metric_data` 和 `topic_event`。
+- `subscribed_topics` 目前只在 `efka_service_channel` 内记录，关闭时实际清理由 `efka_subscription:unsubscribe_all/1` 完成。
+- `efka_subscription` 当前基于列表扫描匹配，订阅规模很大时需要后续优化为索引结构。

docs/todo.md

@@ -0,0 +1,50 @@
+# TODO
+
+## 协议层
+
+### 高优先级
+
+- 将 `message.proto` 收敛为 `efka` 与 `iot` 共用的一份单一事实来源，避免协议定义漂移和双端重复维护。
+
+### 中优先级
+
+- 增强 protobuf 映射和 Docker JSON 生成相关的自动化契约测试。
+
+## Docker 层
+
+### 高优先级
+
+- 将 `docker_commands` 拆分为更清晰的层次：Docker HTTP 客户端、Docker JSON 构造层、以及 efka 本地补丁逻辑。
+- 明确 `docker_task_reporter` 的投递语义；当前只有内存缓冲，还缺少持久化能力和队列上限控制。
+
+### 中优先级
+
+- 丰富 `docker_deploy_manager` 的任务跟踪状态，把部署元数据、耗时、阶段、失败原因等信息纳入统一管理，避免后续再回头重构。
+- 统一 `docker_container_service` 的返回结构，避免调用方分支处理 `ok | {ok, binary()} | {error, binary()}` 这种混合形式。
+- 消除 `docker_commands` 中重复的 Docker HTTP 响应解析逻辑。
+
+### 低优先级
+
+- 对于像 `docker_events.erl` 这样当前未启用的模块，要么清理掉，要么补充说明其保留原因。
+
+## 订阅与通道层
+
+### 高优先级
+
+- 将 `efka_subscription` 当前基于列表扫描的匹配方式替换为更可扩展的索引结构，例如 trie 或基于 ETS 的索引。
+
+### 中优先级
+
+- 继续收敛 `efka_iot_client` 的职责，将传输层状态管理与 request/cast 协议处理进一步拆开。
+- 评估并落实 `efka_service_channel` 中 `subscribed_topics` 集合的用途；如果只是被动保存状态，则应简化。
+
+### 低优先级
+
+- 如果 `efka_iot_client` 的缓存指标量继续增长，可以为缓存刷出增加批量发送或节流机制。
+
+## 基础设施层
+
+### 低优先级
+
+- 如果上传相关能力已经不再属于当前运行时，移除或补充说明残留的启动逻辑，例如 `ensure_upload_dir/0`。
+- 评估是否将 `efka_logger` 与 OTP `logger` 统一，避免长期维护两套日志链路。

message_pb.proto

@@ -19,23 +19,34 @@ message AuthReply {
 // service_id主动订阅消息, 基于广播通讯
 message Pub {
     string topic = 1;
-    string content = 2;
+    bytes content = 2;
+}
+
+message Command {
+    string command_type = 1;
+    bytes command = 2;
 }
 
 ///// 服务器主动推送的消息
 
-message AsyncCallReply {
-    // 0: 表示失败，1: 成功
-    uint32 code = 1;
-    string result = 2;
-    string message = 3;
+// 部署逻辑
+message RPCDeploy {
+    uint32 task_id = 1;
+    // json
+    string config = 2;
 }
 
-// 部署逻辑
-message Deploy {
-    uint32 task_id = 1;
-    string service_id = 2;
-    string tar_url = 3;
+message RPCStartContainer {
+    string container_name = 1;
+}
+
+message RPCStopContainer {
+    string container_name = 1;
+}
+
+message RPCConfigContainer {
+    string container_name = 1;
+    bytes config = 2;
 }
 
 // 获取task的logs
@@ -43,18 +54,11 @@ message FetchTaskLog {
     uint32 task_id = 1;
 }
 
-// 需要响应, 云端主动发起的调用; 提供给用户
-message Invoke {
-    string service_id = 1;
-    string payload = 2;
-    uint32 timeout = 3;
-}
-
 // 参数配置
-message PushServiceConfig {
-    string service_id = 1;
-    string config_json = 2;
-    uint32 timeout = 3;
+message ContainerConfig {
+    string container_name = 1;
+    // 任意的数据格式
+    bytes config = 2;
 }
 
 /////// EFKA主动上报的消息类型
@@ -63,8 +67,15 @@ message PushServiceConfig {
 message Data {
     string service_id = 1;
     string device_uuid = 2;
+    string route_key = 3;
     // measurement[,tag_key=tag_value...] field_key=field_value[,field_key2=field_value2...] [timestamp]
-    string metric = 3;
+    bytes metric = 4;
+}
+
+message Event {
+    string service_id = 1;
+    uint32 event_type = 2;
+    string params = 3;
 }
 
 //#{<<"adcode">> => 0,<<"boot_time">> => 18256077,<<"city">> => <<>>,
@@ -97,24 +108,4 @@ message Ping {
     repeated int32 memory = 12;
     // 接口信息的定义: 每个接口的信息, 采用json格式传输，没有办法提前定义
     string interfaces = 13;
-}
-
-// Inform消息
-message ServiceInform {
-    string service_id = 1;
-    string props = 2;
-    uint32 status = 3;
-    uint32 timestamp = 4;
-}
-
-message Event {
-    string service_id = 1;
-    uint32 event_type = 2;
-    string params = 3;
-}
-
-// 告警信息
-message Alarm {
-    string service_id = 1;
-    string params = 2;
 }

rebar.config

@@ -1,11 +1,15 @@
 {erl_opts, [debug_info]}.
+
+{project_app_dirs, ["apps/*"]}.
+
+{plugins, [
+    {rebar3_gpb_plugin, ".*", {git, "https://github.com/lrascao/rebar3_gpb_plugin.git", {tag, "2.23.8"}}}
+]}.
+
 {deps, [
     {sync, ".*", {git, "https://github.com/rustyio/sync.git", {branch, "master"}}},
-    {jiffy, ".*", {git, "https://github.com/davisp/jiffy.git", {tag, "1.1.2"}}},
-    {gpb, ".*", {git, "https://github.com/tomas-abrahamsson/gpb.git", {tag, "4.20.0"}}},
-    {jiffy, ".*", {git, "https://github.com/davisp/jiffy.git", {tag, "1.1.1"}}},
-    {parse_trans, ".*", {git, "https://github.com/uwiger/parse_trans", {tag, "3.0.0"}}},
-    {lager, ".*", {git,"https://github.com/erlang-lager/lager.git", {tag, "3.9.2"}}}
+    {cowboy, ".*", {git, "https://github.com/ninenines/cowboy.git", {tag, "2.10.0"}}},
+    {gun, ".*", {git, "https://github.com/ninenines/gun.git", {tag, "2.2.0"}}}
 ]}.
 
 {relx, [{release, {efka, "0.1.0"},
@@ -38,11 +42,4 @@
                      ]
             }]}]}.
 
-{project_plugins, [
-    %% 或从 Git 仓库拉取最新版本
-    {pc, {git, "https://github.com/blt/port_compiler.git", {tag, "v1.15.0"}}}
-]}.
-
-{erl_opts, [{parse_transform,lager_transform}]}.
-
-{rebar_packages_cdn, "https://hexpm.upyun.com"}.
+{rebar_packages_cdn, "https://hexpm.upyun.com"}.