fix channel

CODE_LOGIC_OVERVIEW.md

@@ -1,288 +0,0 @@
-# EFKA 当前代码逻辑梳理
-
-> 本文档记录当前阅读代码后对项目逻辑的理解，主要基于 `apps/efka/src/`、`apps/efka/include/`、`config/` 和 protobuf 生成文件。尚未覆盖每个边界条件和全部测试行为。
-
-## 1. 项目定位
-
-这个项目是一个 Erlang/OTP 应用，核心作用像一个边缘侧 agent：
-
-- 对本机微服务开放 WebSocket 接入，微服务注册、订阅 topic、上报指标。
-- 与上游服务器建立 TLS 长连接，完成鉴权、上报数据、接收远程命令。
-- 通过 Docker API 管理本机容器，包括部署、启动、停止、杀掉、删除、更新配置和列表查询。
-- 用 DETS 做少量本地持久化，包括服务状态和离线缓存包。
-
-依赖主要有：
-
-- `cowboy`：本地 WebSocket server。
-- `gun`：通过 Unix Socket 访问 Docker HTTP API。
-- `ssl`：连接上游 TLS server。
-- OTP `json` 模块：JSON 编解码。
-- `gpb`：protobuf 代码生成。
-
-## 2. 启动流程
-
-应用入口是 `efka_app:start/2`。
-
-启动步骤：
-
-1. 设置 Erlang IO 编码为 unicode。
-2. 设置 `fullsweep_after`，试图加快进程内存回收。
-3. 启动 Cowboy WebSocket server。
-4. 启动顶层 supervisor `efka_sup`。
-
-WebSocket server：
-
-- 从 `config/sys.config` 读取 `efka.websocket_server`。
-- 监听 `/ws`。
-- 请求处理模块是 `efka_service_channel`。
-
-`efka_sup` 启动的子进程：
-
-- `efka_logger`：部署日志落盘。
-- `efka_service_sup`：动态管理每个已注册微服务对应的 `efka_service` 进程。
-- `efka_service_model`：DETS 服务状态表。
-- `efka_subscription`：本地 topic 订阅中心。
-- `efka_iot_client`：连接上游 TLS server 的状态机。
-
-`docker` application 启动的子进程：
-
-- `docker_task_reporter`：部署任务事件缓冲和转发。
-- `docker_deploy_manager`：部署任务管理器。
-
-`docker_events` 代码存在于 `apps/docker`，但目前默认不会启动。
-
-## 3. 两条通信链路
-
-项目里有两条主要通信链路。
-
-### 3.1 微服务到 EFKA：WebSocket
-
-本机微服务连接 `ws://<host>:18080/ws`。
-
-协议处理在 `efka_service_channel`：
-
-- WebSocket 收到 `ping` 直接回复 `pong`。
-- 收到 binary 帧时，第一个字节是帧类型。
-- `FRAME_REQUEST = 0x01`，用 `efka_service_pb` 解码为 `ServiceRequest`。
-- `FRAME_CAST = 0x03`，用 `efka_service_pb` 解码为 `ServiceCast`。
-- 回复时使用 `FRAME_REPLY = 0x02`。
-
-支持的微服务请求：
-
-- `register`：注册服务。
-- `subscribe`：订阅 topic。
-
-支持的微服务 cast：
-
-- `metric_data`：上报指标数据。
-
-注册流程：
-
-1. 微服务发送 `ServiceRequest.Register`，包含 `service_id`。
-2. `efka_service_channel` 调用 `efka_service_sup:start_service(ServiceId)`。
-3. 如果该服务进程不存在，则动态启动一个 `efka_service`；如果已存在，复用已有 pid。
-4. `efka_service_channel` 调用 `efka_service:attach_channel(ServicePid, self())` 绑定当前 WebSocket channel。
-5. `efka_service` 只允许绑定一个存活 channel；已有 channel 存活时返回 `channel exists`。
-6. 注册成功后写入 `efka_service_model`，服务状态设为 running。
-7. channel 退出时，取消全部订阅，并把服务状态改为 stopped。
-
-指标上报流程：
-
-1. 微服务发送 `ServiceCast.MetricData`。
-2. `efka_service_channel` 调用 `efka_service:metric_data(ServicePid, RouteKey, Metric)`。
-3. `efka_service` 转发给 `efka_iot_client:metric_data(RouteKey, Metric)`。
-4. `efka_iot_client` 如果处于 activated 状态，直接发给上游；否则写入持久化 outbox。
-
-### 3.2 EFKA 到上游：TLS 长连接
-
-上游连接逻辑在 `efka_iot_client`，它是一个 `gen_statem`。
-
-状态包括：
-
-- `disconnected`：未连接。
-- `auth`：已连接，等待鉴权响应。
-- `restricted`：鉴权受限，不能正常推送数据，但可以接收部分授权命令。
-- `activated`：已激活，可正常收发。
-
-连接流程：
-
-1. 初始化后立即触发 `create_transport`。
-2. 读取 `efka.tls_server_address`，用 `ssl:connect/4` 建立 TLS 连接。
-3. TLS socket 使用 `{packet, 4}` 和 `{active, true}`。
-4. 连接成功后发送 `AuthRequest`。
-5. 收到鉴权成功 reply 后进入 `activated`，并触发 outbox 刷出。
-6. 连接或鉴权失败时关闭 socket，5 秒后重连。
-
-上游协议同样用第一个字节区分帧类型：
-
-- `FRAME_REQUEST = 0x01`
-- `FRAME_REPLY = 0x02`
-- `FRAME_CAST = 0x03`
-
-但 protobuf 使用的是 `message_pb`，不是微服务侧的 `efka_service_pb`。
-
-`efka_iot_client` 上报的内容：
-
-- `metric_data`：业务指标数据，activated 时实时发送，否则进入持久化 outbox。
-- `task_event_stream`：Docker 部署任务流式日志，只在 activated 时发送。
-- `close_task_event_stream`：任务结束事件，只在 activated 时发送。
-
-`efka_iot_client` 接收的内容：
-
-- `RequestFrame.container_request`：远程容器管理请求，交给 `docker_container_service`。
-- `CastFrame.command`：授权命令，`COMMAND_AUTH` 用于切换鉴权/受限状态。
-- `CastFrame.pub`：上游发布的 topic 消息，交给 `efka_subscription:publish/3`。
-
-## 4. 本地 Pub/Sub 逻辑
-
-`efka_subscription` 是本地订阅中心。
-
-订阅：
-
-- `efka_service_channel` 收到微服务 `subscribe` 后调用 `efka_subscription:subscribe(Topic, self())`。
-- 同一个 pid 对同一个 topic 重复订阅会被忽略。
-- 首次看到某个 subscriber pid 时，会 monitor 该 pid；pid 退出后自动移除订阅。
-
-topic 匹配规则：
-
-- `/` 分隔 topic components。
-- `*` 表示单级匹配。
-- `+` 表示多级匹配，但只能出现在末尾。
-- 完全匹配优先级最高，不过当前代码里的 `order` 字段只是计算并保存，没有实际排序使用。
-
-发布：
-
-- 上游发来的 `CastFrame.Pub` 会调用 `efka_subscription:publish(Topic, Qos, Content)`。
-- 如果匹配到订阅者，向对应 `efka_service_channel` 发送 `{topic_broadcast, Topic, Content}`。
-- `efka_service_channel` 再编码为 `ServiceCast.TopicEvent` 推送给微服务。
-- 如果没有订阅者且 `Qos != 0`，消息会暂存在 `remaining_messages`。
-- 新订阅建立时，会尝试把匹配的 remaining messages 补发给该订阅者。
-
-## 5. Docker 容器管理链路
-
-上游发来的容器请求最终进入 `docker_container_service:handle_request/1`。
-
-支持动作：
-
-- `list`：列出容器。
-- `deploy`：部署容器。
-- `start`：启动容器。
-- `stop`：停止容器。
-- `kill`：发送 kill 信号。
-- `remove`：删除容器。
-- `config`：更新容器配置文件。
-
-简单操作：
-
-- `start/stop/kill/remove/list` 直接调用 `docker_commands`。
-- `docker_commands` 通过短生命周期普通 `docker_client` 进程访问 `/var/run/docker.sock`。
-- HTTP 客户端用 `gun:open_unix/2`。
-- 流式请求使用 `{docker_client, Ref, Event}` 消息返回，事件包括 `{response, Status, Headers}`、`{data, Bin}`、`done` 和 `{error, Reason}`。
-
-部署操作：
-
-1. `docker_container_service` 收到 `deploy` 后调用 `docker_deploy_manager:deploy(TaskId, Params)`。
-2. `docker_deploy_manager` 根据 `docker.root_dir` 和参数里的 `container_name/container_dir` 确保容器目录存在。
-3. 它启动一个独立 `docker_deployer` 进程，并 monitor 该部署进程。
-4. `docker_deployer` 执行实际部署步骤。
-5. 部署过程事件写入 `docker_task_reporter`。
-6. `docker_task_reporter` 在上游连接 activated 时把事件转给 `efka_iot_client`；未激活时保留队列并定时重试。
-
-`docker_deployer` 当前部署步骤：
-
-1. 上报“开始部署容器”。
-2. 调用 `ensure_container_absent/2`，但当前实现只是上报“开始创建容器”，没有真正删除旧容器。
-3. 规范化镜像名，没有 tag 时补 `:latest`。
-4. 调用 Docker API 拉取镜像，`docker_deployer` 按 `Ref` 接收 `docker_client` 流式消息并上报拉取过程。
-5. 创建容器。
-6. 创建空的 `service.conf` 配置文件。
-7. 写部署摘要到 `efka_logger`。
-8. 关闭任务事件流，状态为 `success` 或 `fail`。
-
-创建容器参数：
-
-- `docker_container_builder` 把 protobuf 的 `DockerCreateOptions` 转成 Docker API JSON。
-- 会自动给容器注入环境变量 `CONTAINER_NAME=<name>`。
-- 会自动添加容器内配置路径 `/usr/local/etc/service.conf`。
-- 会自动把宿主机 `service.conf` bind 到容器内 `/usr/local/etc/service.conf`。
-
-配置更新：
-
-- `docker_container_service:update_container_config/2` 根据容器名找到目录。
-- 写入该目录下的 `service.conf`。
-- 容器目录映射由 `docker_helper` 通过 `.container_dir` 指针文件维护。
-
-## 6. 本地持久化
-
-### 6.1 `efka_service_model`
-
-使用 DETS 表 `service`。
-
-文件位置来自 `efka.dets_dir`，默认配置是：
-
-```erlang
-"/usr/local/code/tmp/dets/service.dets"
-```
-
-记录结构在 `apps/efka/include/efka_tables.hrl`：
-
-- `service_id`
-- `container_name`
-- `meta_data`
-- `status`
-- `create_ts`
-- `update_ts`
-
-用途：
-
-- 注册成功时插入或更新服务。
-- channel 关闭时把服务状态改成 stopped。
-- 支持查询所有服务、运行中服务和单个服务状态。
-
-### 6.2 `efka_iot_outbox`
-
-使用 append-only log 文件和 metadata 文件。
-
-作用：
-
-- 当 `efka_iot_client` 不在 activated 状态时，把待上报 packet 缓存下来。
-- `efka_iot_client` 激活后循环 `next -> send -> ack` 刷出。
-- 所有记录 ack 后会截断 log，下一轮从 seq 1 重新开始。
-
-## 7. 日志
-
-有两套日志：
-
-- OTP logger：在 `config/sys.config` 里配置到 console 和 `log/debug.log`。
-- `efka_logger`：自定义部署日志，按日期写到 `code:root_dir() ++ "/log/"` 下。
-
-`efka_logger` 主要用于部署任务摘要和失败原因。
-
-## 8. 配置项
-
-关键配置来自 `config/sys.config`：
-
-- `docker.root_dir`：容器相关目录根路径。
-- `dets_dir`：DETS 文件目录。
-- `websocket_server`：本地 WebSocket 监听配置。
-- `tls_server_address`：上游 TLS server 地址。
-- `auth`：上游鉴权信息。
-
-注意：
-
-- `efka_service_model` 打开 DETS 前假设 `dets_dir` 已存在，代码里没有显式创建目录。
-- `docker_client` 固定使用 `/var/run/docker.sock`，每次请求内部由短生命周期普通进程执行 open/request/close。
-
-## 9. 当前看到的几个注意点
-
-- `README.md` 描述的是 JSON-RPC 风格 WebSocket，但当前代码实际处理的是 binary protobuf 帧；README 可能已经过期。
-- `efka_service_channel` 里 `register` 只使用 `service_id`，没有使用 README 中提到的 `meta_data/container_name`。
-- `efka_iot_client:send_result_reply/3` 和 `send_error_reply/3` 编码 `ReplyFrame` 后没有加 `FRAME_REPLY` 前缀；接收侧是否期望裸 protobuf 需要确认。
-- `docker_deployer:ensure_container_absent/2` 当前没有真正确保旧容器不存在，只是上报日志。
-- `efka_subscription` 计算了 topic `order`，但匹配广播时没有使用优先级排序。
-- `docker_commands` 部分错误响应解码没有统一使用 `[return_maps]`，有些分支可能匹配不到 map。
-- `docker_events` 存在但未启动，且使用 shell 命令 `docker events`，与其他 Docker API 访问方式不同。
-
-## 10. 一句话主流程
-
-微服务通过 WebSocket 注册到 EFKA，本地 channel 把指标交给对应 `efka_service`，再由 `efka_iot_client` 通过 TLS 上报给上游；上游通过同一条 TLS 连接下发 pub/sub 消息和容器管理请求，pub/sub 再广播回本地微服务，容器请求则通过 Docker Unix Socket 在本机执行。

README.md

@@ -1,110 +1,13 @@
-# ws_channel 模块 API 文档与交互逻辑
+efka
+=====
 
-## 注意websocket的数据格式为: text
+An OTP application
 
-## 一、模块概述
+1. 先解决数据的上行问题
-`ws_channel` 是基于 Erlang + Cowboy WebSocket 实现的 MQTT 相关交互模块，主要用于服务注册、主题订阅、指标数据上报、事件发送及消息广播等功能，通过 WebSocket 协议实现客户端与服务端的实时双向通信。
+2. todo list
+    要解决连接断开重新连接的问题 !!!
 
+Build
+-----
 
-## 三、核心 API 方法
+    $ rebar3 compile
-客户端通过发送 **JSON 格式文本消息** 与服务端交互，消息格式遵循 JSON-RPC 风格（包含 `id`、`method`、`params` 字段）。
-
-
-### 1. 服务注册（register）
-#### 功能
-注册服务并建立客户端与服务进程的关联，是后续操作（订阅、上报数据等）的前提。
-
-#### 请求格式
-```json
-{
-  "id": <整数，请求唯一标识>,
-  "method": "register",
-  "params": {
-    "service_id": <二进制，服务唯一标识，必填>,
-    "meta_data": <映射，服务元数据，可选>,
-    "container_name": <二进制，容器名称，可选>
-  }
-}
-```
-
-#### 响应格式
-- 成功响应：
-  ```json
-  {
-    "id": <与请求id一致>,
-    "result": "ok"
-  }
-  ```
-- 失败处理：服务端直接关闭连接（因 `attach_channel` 失败）
-
-### 2. 主题订阅（subscribe）
-#### 功能
-订阅指定主题，后续可接收该主题的广播消息。
-
-#### 请求格式
-```json
-{
-  "id": <整数，请求唯一标识>,
-  "method": "subscribe",
-  "params": {
-    "topic": <二进制，订阅的主题名称，必填>
-  }
-}
-```
-
-#### 响应格式
-- 成功响应：
-  ```json
-  {
-    "id": <与请求id一致>,
-    "result": "ok"
-  }
-  ```
-- 失败响应：
-  ```json
-  {
-    "id": <与请求id一致>,
-    "error": {
-      "code": -1,
-      "message": "错误描述"
-    }
-  }
-  ```
-
-#### 处理逻辑
-通过 `efka_subscription:subscribe(Topic, self())` 完成订阅，订阅成功后客户端会收到该主题的广播消息。
-
-
-### 3. 指标数据上报（metric_data）
-#### 功能
-向服务进程上报设备指标数据。
-
-#### 请求格式
-```json
-{
-  "method": "metric_data",
-  "params": {
-    "route_key": <二进制，路由键，必填>,
-    "metric": <指标数据，必填>
-  }
-}
-```
-
-#### 响应处理
-服务端接收后无返回消息（处理逻辑：`efka_service:metric_data(ServicePid, DeviceUUID, RouteKey, Metric)`）
-
-## 五、基础交互协议
-1. **Ping/Pong 心跳**：
-    - 客户端发送 `ping` 消息
-    - 服务端回复 `pong` 消息，维持连接
-
-2. **未知消息处理**：
-    - 客户端发送未定义格式的消息时，服务端记录错误并关闭连接
-
-## 七、典型交互流程
-1. 客户端发起 WebSocket 连接
-2. 客户端发送 `register` 请求完成注册
-3. 客户端发送 `subscribe` 请求订阅目标主题
-4. 客户端通过 `metric_data` 上报指标数据 / 通过 `event` 发送事件
-5. 服务端向客户端推送已订阅主题的消息（`publish` 方法）
-6. 连接关闭（主动断开或异常终止）

apps/docker/src/docker.app.src

@@ -1,15 +0,0 @@
-{application, docker,
- [{description, "Docker integration application"},
-  {vsn, "0.1.0"},
-  {registered, []},
-  {mod, {docker_app, []}},
-  {applications,
-   [
-    gun,
-    kernel,
-    stdlib
-   ]},
-  {modules, []},
-  {licenses, ["Apache-2.0"]},
-  {links, []}
- ]}.

apps/docker/src/docker_app.erl

@@ -1,18 +0,0 @@
-%%%-------------------------------------------------------------------
-%% @doc Docker application entry point.
-%% @end
-%%%-------------------------------------------------------------------
-
--module(docker_app).
-
--behaviour(application).
-
--export([start/2, stop/1]).
-
--spec start(term(), term()) -> {ok, pid()} | {error, term()}.
-start(_StartType, _StartArgs) ->
-    docker_sup:start_link().
-
--spec stop(term()) -> ok.
-stop(_State) ->
-    ok.

apps/docker/src/docker_client.erl

@@ -1,292 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc
-%%% Short-lived Docker API client process.
-%%%
-%%% Each request owns one process and one gun Unix socket connection.
-%%% The process exits after the Docker API response finishes.
-%%% @end
-%%%-------------------------------------------------------------------
--module(docker_client).
-
-%% API
--export([request/4, start_stream/5, start_stream_no_timeout/5]).
-
--define(DOCKER_SOCKET, "/var/run/docker.sock").
--define(RESPONSE_TIMEOUT, 5000).
--define(BODY_TIMEOUT, 10000).
--define(STREAM_BODY_TIMEOUT, 30000).
--define(CLIENT_TIMEOUT, 60000).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec request(Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
-    {ok, StatusCode :: integer(), RespHeaders :: proplists:proplist(), RespBody :: binary()} | {error, any()}.
-request(Method, Path, Body, Headers) when is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
-    Owner = self(),
-    Ref = make_ref(),
-    {Pid, MRef} = spawn_monitor(fun() -> request_worker(Owner, Ref, Method, Path, Body, Headers) end),
-    receive
-        {docker_client, Ref, {result, Result}} ->
-            erlang:demonitor(MRef, [flush]),
-            Result;
-        {'DOWN', MRef, process, Pid, Reason} ->
-            {error, {client_down, Reason}}
-    after ?CLIENT_TIMEOUT ->
-        exit(Pid, shutdown),
-        erlang:demonitor(MRef, [flush]),
-        {error, timeout}
-    end.
-
--spec start_stream(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
-    {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}.
-start_stream(Owner, Method, Path, Body, Headers)
-  when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
-    Ref = make_ref(),
-    {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, ?RESPONSE_TIMEOUT, ?STREAM_BODY_TIMEOUT) end),
-    {ok, Ref, Pid, MRef}.
-
--spec start_stream_no_timeout(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
-    {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}.
-start_stream_no_timeout(Owner, Method, Path, Body, Headers)
-  when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
-    Ref = make_ref(),
-    {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, infinity, infinity) end),
-    {ok, Ref, Pid, MRef}.
-
-%%%===================================================================
-%%% Worker functions
-%%%===================================================================
-
--spec request_worker(pid(), reference(), string(), string(), binary(), list()) -> ok.
-request_worker(Owner, Ref, Method, Path, Body, Headers) ->
-    Result = do_request(Method, Path, Body, Headers),
-    send_owner(Owner, Ref, {result, Result}).
-
--spec stream_worker(pid(), reference(), string(), string(), binary(), list(), timeout(), timeout()) -> ok.
-stream_worker(Owner, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) ->
-    OwnerRef = erlang:monitor(process, Owner),
-    _Result = do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout),
-    erlang:demonitor(OwnerRef, [flush]),
-    ok.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec do_request(string(), string(), binary(), list()) ->
-    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
-do_request(Method, Path, Body, Headers) ->
-    case open_connection() of
-        {ok, ConnPid} ->
-            try
-                StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
-                receive_response(ConnPid, StreamRef)
-            after
-                close_connection(ConnPid)
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec do_stream(pid(), reference(), reference(), string(), string(), binary(), list(), timeout(), timeout()) ->
-    ok | {error, binary()}.
-do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) ->
-    case open_connection() of
-        {ok, ConnPid} ->
-            try
-                StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
-                receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout)
-            after
-                close_connection(ConnPid)
-            end;
-        {error, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason}
-    end.
-
--spec receive_response(pid(), reference()) ->
-    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
-receive_response(ConnPid, StreamRef) ->
-    receive
-        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} ->
-            receive_body(ConnPid, StreamRef, Status, Headers, <<>>);
-        {gun_response, ConnPid, StreamRef, fin, Status, Headers} ->
-            {ok, Status, Headers, <<>>};
-        {gun_error, ConnPid, StreamRef, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_error, ConnPid, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_down, ConnPid, _, Reason, _} ->
-            {error, {http_closed, Reason}}
-    after ?RESPONSE_TIMEOUT ->
-        {error, timeout}
-    end.
-
--spec receive_body(pid(), reference(), integer(), proplists:proplist(), iodata()) ->
-    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
-receive_body(ConnPid, StreamRef, Status, Headers, Acc) ->
-    receive
-        {gun_data, ConnPid, StreamRef, fin, Data} ->
-            Body = iolist_to_binary([Acc, Data]),
-            {ok, Status, Headers, Body};
-        {gun_data, ConnPid, StreamRef, nofin, Data} ->
-            receive_body(ConnPid, StreamRef, Status, Headers, [Acc, Data]);
-        {gun_error, ConnPid, StreamRef, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_error, ConnPid, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_down, ConnPid, _, Reason, _} ->
-            {error, {http_closed, Reason}}
-    after ?BODY_TIMEOUT ->
-        {error, timeout}
-    end.
-
--spec receive_stream_response(pid(), reference(), reference(), pid(), reference(), timeout(), timeout()) -> ok | {error, binary()}.
-receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout) ->
-    receive
-        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} when Status >= 200, Status < 300 ->
-            send_owner(Owner, Ref, {response, Status, Headers}),
-            receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout);
-        {gun_response, ConnPid, StreamRef, fin, Status, Headers} when Status >= 200, Status < 300 ->
-            send_owner(Owner, Ref, {response, Status, Headers}),
-            send_owner(Owner, Ref, done),
-            ok;
-        {gun_response, ConnPid, StreamRef, nofin, Status, _Headers} ->
-            receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, <<>>, stream_error_body_timeout(BodyTimeout));
-        {gun_response, ConnPid, StreamRef, fin, Status, _Headers} ->
-            Reason = http_status_error(Status, <<>>),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, StreamRef, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_down, ConnPid, _, Reason0, _} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
-            {error, format_error({owner_down, Reason0})}
-    after ResponseTimeout ->
-        send_owner(Owner, Ref, {error, <<"处理超时"/utf8>>}),
-        {error, <<"timeout">>}
-    end.
-
--spec receive_stream_body(pid(), reference(), reference(), pid(), reference(), timeout()) -> ok | {error, binary()}.
-receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout) ->
-    receive
-        {gun_data, ConnPid, StreamRef, fin, Data} ->
-            maybe_send_stream_data(Owner, Ref, Data),
-            send_owner(Owner, Ref, done),
-            ok;
-        {gun_data, ConnPid, StreamRef, nofin, Data} ->
-            maybe_send_stream_data(Owner, Ref, Data),
-            receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout);
-        {gun_error, ConnPid, StreamRef, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_down, ConnPid, _, Reason0, _} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
-            {error, format_error({owner_down, Reason0})}
-    after BodyTimeout ->
-        send_owner(Owner, Ref, {error, <<"timeout">>}),
-        {error, <<"timeout">>}
-    end.
-
--spec receive_stream_error_body(pid(), reference(), reference(), pid(), reference(), integer(), iodata(), timeout()) ->
-    {error, binary()}.
-receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, Acc, BodyTimeout) ->
-    receive
-        {gun_data, ConnPid, StreamRef, fin, Data} ->
-            Reason = http_status_error(Status, iolist_to_binary([Acc, Data])),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_data, ConnPid, StreamRef, nofin, Data} ->
-            receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, [Acc, Data], BodyTimeout);
-        {gun_error, ConnPid, StreamRef, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_down, ConnPid, _, Reason0, _} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
-            {error, format_error({owner_down, Reason0})}
-    after BodyTimeout ->
-        Reason = http_status_error(Status, iolist_to_binary(Acc)),
-        send_owner(Owner, Ref, {error, Reason}),
-        {error, Reason}
-    end.
-
--spec stream_error_body_timeout(timeout()) -> timeout().
-stream_error_body_timeout(infinity) ->
-    infinity;
-stream_error_body_timeout(_BodyTimeout) ->
-    ?BODY_TIMEOUT.
-
--spec open_connection() -> {ok, pid()} | {error, any()}.
-open_connection() ->
-    case gun:open_unix(?DOCKER_SOCKET, #{}) of
-        {ok, ConnPid} ->
-            case gun:await_up(ConnPid) of
-                {ok, _} ->
-                    {ok, ConnPid};
-                {error, Reason} ->
-                    close_connection(ConnPid),
-                    {error, Reason}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec close_connection(pid()) -> ok.
-close_connection(ConnPid) ->
-    catch gun:close(ConnPid),
-    ok.
-
--spec send_owner(pid(), reference(), term()) -> ok.
-send_owner(Owner, Ref, Event) ->
-    Owner ! {docker_client, Ref, Event},
-    ok.
-
--spec maybe_send_stream_data(pid(), reference(), iodata()) -> ok.
-maybe_send_stream_data(Owner, Ref, Data) ->
-    case iolist_to_binary(Data) of
-        <<>> ->
-            ok;
-        DataBin ->
-            send_owner(Owner, Ref, {data, DataBin})
-    end.
-
--spec http_status_error(integer(), binary()) -> binary().
-http_status_error(Status, <<>>) ->
-    iolist_to_binary(io_lib:format("docker http status ~B", [Status]));
-http_status_error(Status, Body) when is_binary(Body) ->
-    StatusBin = integer_to_binary(Status),
-    <<"docker http status ", StatusBin/binary, ": ", Body/binary>>.
-
--spec format_error(term()) -> binary().
-format_error(Reason) when is_binary(Reason) ->
-    Reason;
-format_error(Reason) ->
-    iolist_to_binary(io_lib:format("~p", [Reason])).

apps/docker/src/docker_commands.erl

@@ -1,295 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 15. 9月 2025 16:11
-%%%-------------------------------------------------------------------
--module(docker_commands).
--author("anlicheng").
-
-%% API
--export([pull_image/1, check_image_exist/1]).
--export([create_container/2, check_container_exist/1, is_container_running/1,
-    start_container/1, stop_container/1, stop_container/2, remove_container/1, remove_container/3, kill_container/1, kill_container/2,
-    get_containers/0]).
-
--spec pull_image(Image :: binary()) -> {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()} | {error, Reason :: any()}.
-pull_image(Image) when is_binary(Image) ->
-    Url = lists:flatten(io_lib:format("/images/create?fromImage=~s", [binary_to_list(Image)])),
-    docker_client:start_stream_no_timeout(self(), "POST", Url, <<>>, []).
-
--spec check_image_exist(Image :: binary()) -> boolean().
-check_image_exist(Image) when is_binary(Image) ->
-    EncodedImage = uri_string:quote(Image),
-    Url = lists:flatten(io_lib:format("/images/~s/json", [binary_to_list(EncodedImage)])),
-    case docker_client:request("GET", Url, <<"">>, []) of
-        {ok, 200, _Headers, _Resp} ->
-            true;
-        {ok, 404, _, _} ->
-            false;
-        _ ->
-            false
-    end.
-
--spec create_container(ContainerDir :: string(), Params :: map()) ->
-    {ok, ContainerId :: binary()} | {error, Reason :: any()}.
-create_container(ContainerDir, #{<<"container_name">> := ContainerName0, <<"create">> := CreateOpts})
-    when is_list(ContainerDir), is_binary(ContainerName0), is_map(CreateOpts) ->
-    Url = lists:flatten(io_lib:format("/containers/create?name=~s", [binary_to_list(ContainerName0)])),
-    Options = docker_container_builder:build_options(ContainerName0, ContainerDir, CreateOpts),
-    display_options(Options),
-
-    Body = iolist_to_binary(json:encode(Options)),
-
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-
-    case docker_client:request("POST", Url, Body, Headers) of
-        {ok, 201, _Headers, Resp} ->
-            case catch json:decode(Resp) of
-                #{<<"Id">> := ContainerId} when is_binary(ContainerId) ->
-                    {ok, ContainerId};
-                _ ->
-                    {error, Resp}
-            end;
-        {ok, _StatusCode, _, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end;
-create_container(_ContainerDir, _Params) ->
-    {error, <<"invalid container params">>}.
-
--spec is_container_running(ContainerId :: binary()) -> boolean().
-is_container_running(ContainerId) when is_binary(ContainerId) ->
-    case inspect_container(ContainerId) of
-        {ok, #{<<"State">> := #{<<"Running">> := Running}}} ->
-            Running;
-        {error, _} ->
-            false
-    end.
-
--spec check_container_exist(ContainerName :: binary()) -> boolean().
-check_container_exist(ContainerName) when is_binary(ContainerName) ->
-    case inspect_container(ContainerName) of
-        {ok, #{<<"Id">> := Id}} when is_binary(Id) ->
-            true;
-        _ ->
-            false
-    end.
-
--spec start_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-start_container(ContainerName) when is_binary(ContainerName) ->
-    Url = lists:flatten(io_lib:format("/containers/~s/start", [binary_to_list(ContainerName)])),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("POST", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, 304, _Headers, _} ->
-            {error, <<"container already started">>};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec stop_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-stop_container(ContainerName) when is_binary(ContainerName) ->
-    stop_container(ContainerName, 0).
-
--spec stop_container(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> ok | {error, Reason :: binary()}.
-stop_container(ContainerName, TimeoutSeconds) when is_binary(ContainerName), is_integer(TimeoutSeconds), TimeoutSeconds >= 0 ->
-    Url = build_stop_container_url(ContainerName, TimeoutSeconds),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("POST", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, 304, _Headers, _} ->
-            {error, <<"container already stopped">>};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec kill_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-kill_container(ContainerName) when is_binary(ContainerName) ->
-    kill_container(ContainerName, <<>>).
-
--spec kill_container(ContainerName :: binary(), Signal :: binary()) -> ok | {error, Reason :: binary()}.
-kill_container(ContainerName, Signal) when is_binary(ContainerName), is_binary(Signal) ->
-    Url = build_kill_container_url(ContainerName, Signal),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("POST", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec remove_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-remove_container(ContainerName) when is_binary(ContainerName) ->
-    remove_container(ContainerName, false, false).
-
--spec remove_container(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> ok | {error, Reason :: binary()}.
-remove_container(ContainerName, Force, RemoveVolumes)
-  when is_binary(ContainerName), is_boolean(Force), is_boolean(RemoveVolumes) ->
-    Url = build_remove_container_url(ContainerName, Force, RemoveVolumes),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("DELETE", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, 304, _Headers, _} ->
-            {error, <<"container already stopped">>};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec get_containers() -> {ok, Containers :: [map()]} | {error, Reason :: binary()}.
-get_containers() ->
-    Url = "/containers/json?all=true",
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("GET", Url, <<>>, Headers) of
-        {ok, 200, _Headers, ContainersBin} ->
-            Containers = json:decode(ContainersBin),
-            {ok, Containers};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec inspect_container(ContainerId :: binary()) -> {ok, Json :: map()} | {error, Error :: any()}.
-inspect_container(ContainerId) when is_binary(ContainerId) ->
-    Url = lists:flatten(io_lib:format("/containers/~s/json", [binary_to_list(ContainerId)])),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("GET", Url, <<>>, Headers) of
-        {ok, 200, _Headers, Resp} ->
-            decode_container_inspect_summary(Resp);
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec decode_container_inspect_summary(binary()) -> {ok, map()} | {error, binary()}.
-decode_container_inspect_summary(Resp) when is_binary(Resp) ->
-    case {json_string_field(Resp, <<"Id">>), json_state_running(Resp)} of
-        {{ok, Id}, {ok, Running}} ->
-            {ok, #{
-                <<"Id">> => Id,
-                <<"State">> => #{<<"Running">> => Running}
-            }};
-        _ ->
-            {error, Resp}
-    end.
-
--spec json_string_field(binary(), binary()) -> {ok, binary()} | error.
-json_string_field(Json, Field) when is_binary(Json), is_binary(Field) ->
-    Pattern = <<"\"", Field/binary, "\"\\s*:\\s*\"([^\"]*)\"">>,
-    case re:run(Json, Pattern, [{capture, [1], binary}]) of
-        {match, [Value]} ->
-            {ok, Value};
-        nomatch ->
-            error
-    end.
-
--spec json_state_running(binary()) -> {ok, boolean()} | error.
-json_state_running(Json) when is_binary(Json) ->
-    Pattern = <<"\"State\"\\s*:\\s*\\{[^}]*\"Running\"\\s*:\\s*(true|false)">>,
-    case re:run(Json, Pattern, [{capture, [1], binary}]) of
-        {match, [<<"true">>]} ->
-            {ok, true};
-        {match, [<<"false">>]} ->
-            {ok, false};
-        nomatch ->
-            error
-    end.
-
--spec display_options(Options :: map()) -> ok.
-display_options(Options) when is_map(Options) ->
-    logger:debug("deploy options: ~p", [iolist_to_binary(json:encode(Options))]),
-    %lists:foreach(fun({K, V}) -> logger:debug("~p => ~p", [K, V]) end, maps:to_list(Options)),
-    ok.
-
--spec build_stop_container_url(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> string().
-build_stop_container_url(ContainerName, 0) ->
-    lists:flatten(io_lib:format("/containers/~s/stop", [binary_to_list(ContainerName)]));
-build_stop_container_url(ContainerName, TimeoutSeconds) ->
-    lists:flatten(io_lib:format("/containers/~s/stop?t=~B", [binary_to_list(ContainerName), TimeoutSeconds])).
-
--spec build_kill_container_url(ContainerName :: binary(), Signal :: binary()) -> string().
-build_kill_container_url(ContainerName, <<>>) ->
-    lists:flatten(io_lib:format("/containers/~s/kill", [binary_to_list(ContainerName)]));
-build_kill_container_url(ContainerName, Signal) ->
-    lists:flatten(io_lib:format("/containers/~s/kill?signal=~s", [binary_to_list(ContainerName), binary_to_list(Signal)])).
-
--spec build_remove_container_url(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> string().
-build_remove_container_url(ContainerName, Force, RemoveVolumes) ->
-    ForceValue = boolean_to_query_value(Force),
-    RemoveVolumesValue = boolean_to_query_value(RemoveVolumes),
-    lists:flatten(io_lib:format("/containers/~s?force=~s&v=~s", [
-        binary_to_list(ContainerName),
-        ForceValue,
-        RemoveVolumesValue
-    ])).
-
--spec boolean_to_query_value(boolean()) -> string().
-boolean_to_query_value(true) ->
-    "true";
-boolean_to_query_value(false) ->
-    "false".

apps/docker/src/docker_commands_tests.erl

@@ -1,321 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 23. 9月 2025 17:23
-%%%-------------------------------------------------------------------
--module(docker_commands_tests).
--author("anlicheng").
-
-%% API
--export([
-    test_all/0,
-    test_pull/0,
-    test_check_image_exist/0,
-    test_check_image_not_exist/0,
-    test_create_container/0,
-    test_create_container_without_create_options/0,
-    test_create_container_patches_options/0,
-    test_check_container_exist/0,
-    test_check_container_not_exist/0,
-    test_start_container/0,
-    test_stop_container/0,
-    test_stop_container_with_timeout/0,
-    test_kill_container/0,
-    test_kill_container_with_signal/0,
-    test_remove_container/0,
-    test_remove_container_with_options/0,
-    test_get_containers/0
-]).
-
--define(TEST_IMAGE, <<"docker.1ms.run/library/nginx:latest">>).
--define(TEST_CMD, [<<"nginx">>, <<"-g">>, <<"daemon off;">>]).
-
--spec test_all() -> ok.
-test_all() ->
-    ok = test_pull(),
-    ok = test_check_image_exist(),
-    ok = test_check_image_not_exist(),
-    ok = test_create_container(),
-    ok = test_create_container_without_create_options(),
-    ok = test_create_container_patches_options(),
-    ok = test_check_container_exist(),
-    ok = test_check_container_not_exist(),
-    ok = test_start_container(),
-    ok = test_stop_container(),
-    ok = test_stop_container_with_timeout(),
-    ok = test_kill_container(),
-    ok = test_kill_container_with_signal(),
-    ok = test_remove_container(),
-    ok = test_remove_container_with_options(),
-    ok = test_get_containers(),
-    ok.
-
--spec test_pull() -> ok.
-test_pull() ->
-    {ok, Ref, Pid, MRef} = docker_commands:pull_image(?TEST_IMAGE),
-    await_pull(Ref, Pid, MRef).
-
--spec await_pull(reference(), pid(), reference()) -> ok.
-await_pull(Ref, Pid, MRef) ->
-    receive
-        {docker_client, Ref, {response, _Status, _Headers}} ->
-            await_pull(Ref, Pid, MRef);
-        {docker_client, Ref, {data, Data}} ->
-            logger:debug("msg is: ~p", [Data]),
-            await_pull(Ref, Pid, MRef);
-        {docker_client, Ref, done} ->
-            erlang:demonitor(MRef, [flush]),
-            ok;
-        {docker_client, Ref, {error, Reason}} ->
-            erlang:demonitor(MRef, [flush]),
-            error({pull_failed, Reason});
-        {'DOWN', MRef, process, Pid, Reason} ->
-            error({pull_client_down, Reason})
-    end.
-
--spec test_check_image_exist() -> ok.
-test_check_image_exist() ->
-    true = docker_commands:check_image_exist(?TEST_IMAGE),
-    ok.
-
--spec test_check_image_not_exist() -> ok.
-test_check_image_not_exist() ->
-    false = docker_commands:check_image_exist(<<"docker.1ms.run/library/not-exists-for-efka-tests:latest">>),
-    ok.
-
--spec test_create_container() -> ok.
-test_create_container() ->
-    Name = test_container_name(<<"create">>),
-    ContainerDir = prepare_container_dir(Name),
-    {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)),
-    true = is_binary(ContainerId),
-    ok.
-
--spec test_create_container_without_create_options() -> ok.
-test_create_container_without_create_options() ->
-    Name = test_container_name(<<"create-default">>),
-    ContainerDir = prepare_container_dir(Name),
-    Options = docker_container_builder:build_options(Name, ContainerDir, undefined),
-    assert_patched_default_options(Name, ContainerDir, Options),
-    ok.
-
--spec test_create_container_patches_options() -> ok.
-test_create_container_patches_options() ->
-    Name = test_container_name(<<"create-patch">>),
-    ContainerDir = prepare_container_dir(Name),
-    Create = #{
-        config => #{
-            image => ?TEST_IMAGE,
-            cmd => ?TEST_CMD,
-            env => [<<"EXISTING_ENV=1">>],
-            volumes => [<<"/data">>]
-        },
-        host_config => #{
-            binds => [<<"/tmp:/tmp">>]
-        }
-    },
-    Params = #{
-        container_name => Name,
-        create => Create
-    },
-    Options = docker_container_builder:build_options(Name, ContainerDir, Create),
-    assert_patched_default_options(Name, ContainerDir, Options),
-    assert_existing_options_preserved(Options),
-    try
-        ok = test_pull(),
-        {ok, _ContainerId} = docker_commands:create_container(ContainerDir, Params),
-        ok
-    after
-        ok
-        %cleanup_container(Name)
-    end.
-
--spec test_check_container_exist() -> ok.
-test_check_container_exist() ->
-    Name = test_container_name(<<"exist">>),
-    with_created_container(Name, fun(_ContainerDir, _ContainerId) ->
-        true = docker_commands:check_container_exist(Name),
-        ok
-    end).
-
--spec test_check_container_not_exist() -> ok.
-test_check_container_not_exist() ->
-    false = docker_commands:check_container_exist(test_container_name(<<"missing">>)),
-    ok.
-
--spec test_start_container() -> ok.
-test_start_container() ->
-    Name = test_container_name(<<"start">>),
-    with_created_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:start_container(Name),
-        true = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_stop_container() -> ok.
-test_stop_container() ->
-    Name = test_container_name(<<"stop">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:stop_container(Name),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_stop_container_with_timeout() -> ok.
-test_stop_container_with_timeout() ->
-    Name = test_container_name(<<"stop-timeout">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:stop_container(Name, 1),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_kill_container() -> ok.
-test_kill_container() ->
-    Name = test_container_name(<<"kill">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:kill_container(Name),
-        timer:sleep(200),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_kill_container_with_signal() -> ok.
-test_kill_container_with_signal() ->
-    Name = test_container_name(<<"kill-signal">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:kill_container(Name, <<"SIGKILL">>),
-        timer:sleep(200),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_remove_container() -> ok.
-test_remove_container() ->
-    Name = test_container_name(<<"remove">>),
-    with_created_container(Name, fun(_ContainerDir, _ContainerId) ->
-        ok = docker_commands:remove_container(Name),
-        false = docker_commands:check_container_exist(Name),
-        ok
-    end, false).
-
--spec test_remove_container_with_options() -> ok.
-test_remove_container_with_options() ->
-    Name = test_container_name(<<"remove-opts">>),
-    with_started_container(Name, fun(_ContainerDir, _ContainerId) ->
-        ok = docker_commands:remove_container(Name, true, false),
-        false = docker_commands:check_container_exist(Name),
-        ok
-    end, false).
-
--spec test_get_containers() -> ok.
-test_get_containers() ->
-    Name = test_container_name(<<"list">>),
-    with_created_container(Name, fun(_ContainerDir, ContainerId) ->
-        {ok, Containers} = docker_commands:get_containers(),
-        logger:debug("list containers: ~p", [Containers]),
-        true = is_list(Containers),
-        true = contains_container(Name, ContainerId, Containers),
-        ok
-    end).
-
--spec with_created_container(binary(), fun((string(), binary()) -> ok)) -> ok.
-with_created_container(Name, Fun) ->
-    with_created_container(Name, Fun, true).
-
--spec with_created_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok.
-with_created_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) ->
-    ContainerDir = prepare_container_dir(Name),
-    try
-        ok = test_pull(),
-        {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)),
-        ok = Fun(ContainerDir, ContainerId)
-    after
-        case Cleanup of
-            true ->
-                cleanup_container(Name);
-            false ->
-                ok
-        end
-    end.
-
--spec with_started_container(binary(), fun((string(), binary()) -> ok)) -> ok.
-with_started_container(Name, Fun) ->
-    with_started_container(Name, Fun, true).
-
--spec with_started_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok.
-with_started_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) ->
-    with_created_container(Name, fun(ContainerDir, ContainerId) ->
-        ok = docker_commands:start_container(Name),
-        ok = Fun(ContainerDir, ContainerId)
-    end, Cleanup).
-
--spec minimal_params(binary()) -> map().
-minimal_params(Name) when is_binary(Name) ->
-    #{
-        container_name => Name,
-        create => #{
-            config => #{
-                image => ?TEST_IMAGE,
-                cmd => ?TEST_CMD
-            }
-        }
-    }.
-
--spec prepare_container_dir(binary()) -> string().
-prepare_container_dir(Name) when is_binary(Name) ->
-    Dir = lists:flatten(io_lib:format("/usr/local/code/efka/~ts/", [Name])),
-    ok = filelib:ensure_dir(Dir ++ "placeholder"),
-    ok = file:write_file(Dir ++ "service.conf", <<>>, [write]),
-    Dir.
-
--spec cleanup_container(binary()) -> ok.
-cleanup_container(Name) when is_binary(Name) ->
-    _ = docker_commands:remove_container(Name, true, false),
-    ok.
-
--spec assert_patched_default_options(binary(), string(), map()) -> ok.
-assert_patched_default_options(Name, ContainerDir, Options)
-  when is_binary(Name), is_list(ContainerDir), is_map(Options) ->
-    ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)),
-    ExpectedBind = <<ConfigFile/binary, ":/usr/local/etc/service.conf">>,
-    #{<<"Env">> := Env,
-        <<"Volumes">> := Volumes,
-        <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options,
-    true = lists:member(<<"CONTAINER_NAME=", Name/binary>>, Env),
-    true = maps:is_key(<<"/usr/local/etc/service.conf">>, Volumes),
-    true = lists:member(ExpectedBind, Binds),
-    ok.
-
--spec assert_existing_options_preserved(map()) -> ok.
-assert_existing_options_preserved(Options) when is_map(Options) ->
-    #{<<"Env">> := Env,
-        <<"Volumes">> := Volumes,
-        <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options,
-    true = lists:member(<<"EXISTING_ENV=1">>, Env),
-    true = maps:is_key(<<"/data">>, Volumes),
-    true = lists:member(<<"/tmp:/tmp">>, Binds),
-    ok.
-
--spec contains_container(binary(), binary(), [map()]) -> boolean().
-contains_container(Name, ContainerId, Containers) when is_binary(Name), is_binary(ContainerId), is_list(Containers) ->
-    lists:any(fun(Container) -> container_matches(Name, ContainerId, Container) end, Containers).
-
--spec container_matches(binary(), binary(), map()) -> boolean().
-container_matches(Name, ContainerId, #{<<"Id">> := Id, <<"Names">> := Names}) when is_binary(Id), is_list(Names) ->
-    lists:member(<<"/", Name/binary>>, Names) orelse has_id_prefix(ContainerId, Id);
-container_matches(_Name, _ContainerId, _Container) ->
-    false.
-
--spec has_id_prefix(binary(), binary()) -> boolean().
-has_id_prefix(ExpectedId, ActualId) when is_binary(ExpectedId), is_binary(ActualId) ->
-    PrefixLen = erlang:min(byte_size(ExpectedId), byte_size(ActualId)),
-    binary:part(ExpectedId, 0, PrefixLen) =:= binary:part(ActualId, 0, PrefixLen).
-
--spec test_container_name(binary()) -> binary().
-test_container_name(Prefix) when is_binary(Prefix) ->
-    Suffix = integer_to_binary(erlang:unique_integer([positive])),
-    <<"efka-test-", Prefix/binary, "-", Suffix/binary>>.

apps/docker/src/docker_container_builder.erl

@@ -1,379 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 21. 4月 2026 15:58
-%%%-------------------------------------------------------------------
--module(docker_container_builder).
--author("anlicheng").
-
-%% API
--export([build_options/3]).
-
--spec build_options(binary(), string(), map() | undefined) -> map().
-build_options(ContainerName, ContainerDir, Create0) when is_binary(ContainerName), is_list(ContainerDir) ->
-    ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)),
-    Create = patch_create_options(ContainerName, ConfigFile, Create0),
-    build_create_options(Create).
-
--spec patch_create_options(binary(), binary(), map() | undefined) -> map().
-patch_create_options(ContainerName, ConfigFile, undefined) ->
-    patch_create_options(ContainerName, ConfigFile, #{});
-patch_create_options(ContainerName, ConfigFile, Create0)
-  when is_binary(ContainerName), is_binary(ConfigFile), is_map(Create0) ->
-    Config = patch_container_config(ContainerName, ensure_container_config(field(Create0, <<"config">>, undefined))),
-    HostConfig = patch_host_config(ConfigFile, ensure_host_config(field(Create0, <<"host_config">>, undefined))),
-    Create0#{
-        <<"config">> => Config,
-        <<"host_config">> => HostConfig
-    }.
-
--spec patch_container_config(binary(), map()) -> map().
-patch_container_config(ContainerName, Config0) when is_binary(ContainerName), is_map(Config0) ->
-    Env0 = [to_binary(EnvItem) || EnvItem <- field(Config0, <<"env">>, [])],
-    Volumes0 = [to_binary(Volume) || Volume <- field(Config0, <<"volumes">>, [])],
-    ConfigVolume = <<"/usr/local/etc/service.conf">>,
-    Envs = add_unique_front([<<"CONTAINER_NAME=", ContainerName/binary>>], Env0),
-    Volumes = add_unique_front([ConfigVolume], Volumes0),
-    Config0#{
-        <<"env">> => Envs,
-        <<"volumes">> => Volumes
-    }.
-
--spec patch_host_config(binary(), map()) -> map().
-patch_host_config(ConfigFile, HostConfig0) when is_binary(ConfigFile), is_map(HostConfig0) ->
-    Binds0 = [to_binary(Bind) || Bind <- field(HostConfig0, <<"binds">>, [])],
-    ConfigBind = <<ConfigFile/binary, ":/usr/local/etc/service.conf">>,
-    HostConfig0#{<<"binds">> => add_unique_front([ConfigBind], Binds0)}.
-
--spec ensure_container_config(map() | undefined) -> map().
-ensure_container_config(undefined) ->
-    #{};
-ensure_container_config(Config) when is_map(Config) ->
-    Config.
-
--spec ensure_host_config(map() | undefined) -> map().
-ensure_host_config(undefined) ->
-    #{};
-ensure_host_config(HostConfig) when is_map(HostConfig) ->
-    HostConfig.
-
--spec build_create_options(map()) -> map().
-build_create_options(Create0) when is_map(Create0) ->
-    Config = field(Create0, <<"config">>, #{}),
-    HostConfig = field(Create0, <<"host_config">>, #{}),
-    Endpoints = networking_config_endpoints(field(Create0, <<"networking_config">>, undefined)),
-    #{
-        <<"Image">> => to_binary(field(Config, <<"image">>, <<>>)),
-        <<"Cmd">> => [to_binary(CommandItem) || CommandItem <- field(Config, <<"cmd">>, [])],
-        <<"Entrypoint">> => [to_binary(EntrypointItem) || EntrypointItem <- field(Config, <<"entrypoint">>, [])],
-        <<"Env">> => [to_binary(EnvItem) || EnvItem <- field(Config, <<"env">>, [])],
-        <<"Labels">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(field(Config, <<"labels">>, #{}))]),
-        <<"Volumes">> => build_volumes(field(Config, <<"volumes">>, [])),
-        <<"User">> => to_binary(field(Config, <<"user">>, <<>>)),
-        <<"WorkingDir">> => to_binary(field(Config, <<"working_dir">>, <<>>)),
-        <<"Hostname">> => to_binary(field(Config, <<"hostname">>, <<>>)),
-        <<"ExposedPorts">> => build_expose(field(Config, <<"exposed_ports">>, [])),
-        <<"NetworkingConfig">> => build_networking_config(Endpoints),
-        <<"Healthcheck">> => build_healthcheck(field(Config, <<"healthcheck">>, undefined)),
-        <<"HostConfig">> => fold_merge([
-            build_binds(field(HostConfig, <<"binds">>, [])),
-            build_network_mode(field(HostConfig, <<"network_mode">>, <<>>)),
-            build_restart(field(HostConfig, <<"restart_policy">>, undefined)),
-            build_privileged(field(HostConfig, <<"privileged">>, false)),
-            build_cap_add_drop(field(HostConfig, <<"cap_add">>, []), field(HostConfig, <<"cap_drop">>, [])),
-            build_devices(field(HostConfig, <<"devices">>, [])),
-            build_resources(
-                field(HostConfig, <<"memory">>, 0),
-                field(HostConfig, <<"memory_reservation">>, 0),
-                field(HostConfig, <<"nano_cpus">>, 0),
-                field(HostConfig, <<"cpu_shares">>, 0)
-            ),
-            build_port_bindings(field(HostConfig, <<"port_bindings">>, [])),
-            build_ulimits(field(HostConfig, <<"ulimits">>, [])),
-            build_tmpfs(field(HostConfig, <<"tmpfs">>, #{})),
-            build_sysctls(field(HostConfig, <<"sysctls">>, #{})),
-            build_extra_hosts(field(HostConfig, <<"extra_hosts">>, []))
-        ])
-    }.
-
--spec networking_config_endpoints(map() | undefined) -> [map()].
-networking_config_endpoints(undefined) ->
-    [];
-networking_config_endpoints(NetworkingConfig) when is_map(NetworkingConfig) ->
-    field(NetworkingConfig, <<"endpoints">>, []).
-
--spec fold_merge([map()]) -> map().
-fold_merge(List) ->
-    lists:foldl(fun maps:merge/2, #{}, List).
-
--spec build_expose([map()]) -> map().
-build_expose(Ports) when is_list(Ports) ->
-    case Ports of
-        [] ->
-            #{};
-        _ ->
-            maps:from_list([{normalize_expose_port(Port), #{}} || Port <- Ports])
-    end.
-
--spec build_volumes([binary() | list() | atom()]) -> map().
-build_volumes(Volumes) when is_list(Volumes) ->
-    case Volumes of
-        [] ->
-            #{};
-        _ ->
-            maps:from_list([{to_binary(ContainerPath), #{}} || ContainerPath <- Volumes])
-    end.
-
--spec build_binds([binary() | list() | atom()]) -> map().
-build_binds(Binds) when is_list(Binds) ->
-    case Binds of
-        [] ->
-            #{};
-        _ ->
-            #{<<"Binds">> => [to_binary(Bind) || Bind <- Binds]}
-    end.
-
--spec build_networking_config([map()]) -> map().
-build_networking_config(Endpoints) when is_list(Endpoints) ->
-    case Endpoints of
-        [] ->
-            #{};
-        _ ->
-            NetCfg = maps:from_list([{to_binary(Name), #{}} || #{<<"name">> := Name} <- Endpoints]),
-            #{<<"EndpointsConfig">> => NetCfg}
-    end.
-
--spec build_network_mode(binary() | list() | atom()) -> map().
-build_network_mode(<<>>) ->
-    #{};
-build_network_mode(NetworkMode) ->
-    #{<<"NetworkMode">> => to_binary(NetworkMode)}.
-
--spec build_healthcheck(map() | undefined) -> map().
-build_healthcheck(undefined) ->
-    #{};
-build_healthcheck(Healthcheck) when is_map(Healthcheck) ->
-    #{
-        <<"Test">> => [to_binary(Item) || Item <- field(Healthcheck, <<"test">>, [])],
-        <<"Interval">> => field(Healthcheck, <<"interval_ns">>, 0),
-        <<"Timeout">> => field(Healthcheck, <<"timeout_ns">>, 0),
-        <<"Retries">> => field(Healthcheck, <<"retries">>, 0)
-    }.
-
--spec build_restart(map() | undefined) -> map().
-build_restart(undefined) ->
-    #{};
-build_restart(RestartPolicy0) when is_map(RestartPolicy0) ->
-    RestartPolicy = #{
-        <<"Name">> => to_binary(field(RestartPolicy0, <<"name">>, <<>>))
-    },
-    case field(RestartPolicy0, <<"maximum_retry_count">>, 0) of
-        0 ->
-            #{<<"RestartPolicy">> => RestartPolicy};
-        RetryCount ->
-            #{<<"RestartPolicy">> => RestartPolicy#{
-                <<"MaximumRetryCount">> => RetryCount
-            }}
-    end.
-
--spec build_privileged(any()) -> map().
-build_privileged(Privileged) ->
-    case to_bool(Privileged) of
-        true ->
-            #{<<"Privileged">> => true};
-        _ ->
-            #{}
-    end.
-
--spec build_cap_add_drop([binary() | list() | atom()], [binary() | list() | atom()]) -> map().
-build_cap_add_drop(Add, Drop) when is_list(Add), is_list(Drop) ->
-    case {Add, Drop} of
-        {[], []} ->
-            #{};
-        _ ->
-            #{
-                <<"CapAdd">> => [to_binary(Item) || Item <- Add],
-                <<"CapDrop">> => [to_binary(Item) || Item <- Drop]
-            }
-    end.
-
--spec build_devices([map()]) -> map().
-build_devices(Devices) when is_list(Devices) ->
-    case Devices of
-        [] ->
-            #{};
-        _ ->
-            DevObjs = [#{
-                <<"PathOnHost">> => to_binary(HostPath),
-                <<"PathInContainer">> => to_binary(ContainerPath),
-                <<"CgroupPermissions">> => device_permissions(Permissions)
-            } || #{
-                <<"path_on_host">> := HostPath,
-                <<"path_in_container">> := ContainerPath,
-                <<"cgroup_permissions">> := Permissions
-            } <- Devices],
-            #{<<"Devices">> => DevObjs}
-    end.
-
--spec build_resources(integer(), integer(), integer(), integer()) -> map().
-build_resources(MemoryBytes, ReservationBytes, NanoCpus, CpuShares) ->
-    HostConfig0 = #{},
-    HostConfig1 = case MemoryBytes of
-        0 ->
-            HostConfig0;
-        _ ->
-            HostConfig0#{<<"Memory">> => MemoryBytes}
-    end,
-    HostConfig2 = case ReservationBytes of
-        0 ->
-            HostConfig1;
-        _ ->
-            HostConfig1#{<<"MemoryReservation">> => ReservationBytes}
-    end,
-    HostConfig3 = case NanoCpus of
-        0 ->
-            HostConfig2;
-        _ ->
-            HostConfig2#{<<"NanoCpus">> => NanoCpus}
-    end,
-    case CpuShares of
-        0 ->
-            HostConfig3;
-        _ ->
-            HostConfig3#{<<"CpuShares">> => CpuShares}
-    end.
-
--spec build_port_bindings([map()]) -> map().
-build_port_bindings(PortBindings) when is_list(PortBindings) ->
-    case PortBindings of
-        [] ->
-            #{};
-        _ ->
-            #{<<"PortBindings">> => fold_port_bindings(PortBindings)}
-    end.
-
--spec fold_port_bindings([map()]) -> map().
-fold_port_bindings(PortBindings) ->
-    lists:foldl(
-        fun(PortBinding, Acc) ->
-            PortKey = port_binding_key(PortBinding),
-            Binding = port_binding_value(PortBinding),
-            Existing = maps:get(PortKey, Acc, []),
-            Acc#{PortKey => Existing ++ [Binding]}
-        end,
-        #{},
-        PortBindings).
-
--spec port_binding_key(map()) -> binary().
-port_binding_key(#{<<"container_port">> := ContainerPort, <<"protocol">> := Protocol}) ->
-    PortBin = integer_to_binary(ContainerPort),
-    ProtocolBin = to_binary(Protocol),
-    case ProtocolBin of
-        <<>> ->
-            <<PortBin/binary, "/tcp">>;
-        <<"tcp">> ->
-            <<PortBin/binary, "/tcp">>;
-        _ ->
-            <<PortBin/binary, "/", ProtocolBin/binary>>
-    end.
-
--spec port_binding_value(map()) -> map().
-port_binding_value(#{<<"host_port">> := HostPort} = PortBinding) ->
-    HostIp = to_binary(field(PortBinding, <<"host_ip">>, <<>>)),
-    #{<<"HostIp">> => HostIp, <<"HostPort">> => integer_to_binary(HostPort)}.
-
--spec build_ulimits([map()]) -> map().
-build_ulimits(Ulimits) when is_list(Ulimits) ->
-    case Ulimits of
-        [] ->
-            #{};
-        _ ->
-            #{<<"Ulimits">> => [#{
-                <<"Name">> => to_binary(Name),
-                <<"Soft">> => Soft,
-                <<"Hard">> => Hard
-            } || #{<<"name">> := Name, <<"soft">> := Soft, <<"hard">> := Hard} <- Ulimits]}
-    end.
-
--spec build_sysctls(map()) -> map().
-build_sysctls(Sysctls) when is_map(Sysctls) ->
-    case maps:size(Sysctls) of
-        0 ->
-            #{};
-        _ ->
-            #{<<"Sysctls">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(Sysctls)])}
-    end.
-
--spec build_tmpfs(map()) -> map().
-build_tmpfs(Tmpfs) when is_map(Tmpfs) ->
-    case maps:size(Tmpfs) of
-        0 ->
-            #{};
-        _ ->
-            #{<<"Tmpfs">> => maps:from_list([{to_binary(Path), to_binary(Options)} || {Path, Options} <- maps:to_list(Tmpfs)])}
-    end.
-
--spec build_extra_hosts([binary() | list() | atom()]) -> map().
-build_extra_hosts(Hosts) when is_list(Hosts) ->
-    case Hosts of
-        [] ->
-            #{};
-        _ ->
-            #{<<"ExtraHosts">> => [to_binary(Host) || Host <- Hosts]}
-    end.
-
--spec normalize_expose_port(map()) -> binary().
-normalize_expose_port(#{<<"container_port">> := Port, <<"protocol">> := Protocol}) ->
-    PortBin = integer_to_binary(Port),
-    ProtocolBin = to_binary(Protocol),
-    case ProtocolBin of
-        <<>> ->
-            <<PortBin/binary, "/tcp">>;
-        <<"tcp">> ->
-            <<PortBin/binary, "/tcp">>;
-        _ ->
-            <<PortBin/binary, "/", ProtocolBin/binary>>
-    end.
-
--spec device_permissions(binary() | list() | atom()) -> binary().
-device_permissions(<<>>) ->
-    <<"rwm">>;
-device_permissions(Permissions) ->
-    to_binary(Permissions).
-
--spec add_unique_front([binary()], [binary()]) -> [binary()].
-add_unique_front([], List) ->
-    List;
-add_unique_front([Item | Rest], List) ->
-    NList = case lists:member(Item, List) of
-        true ->
-            List;
-        false ->
-            [Item | List]
-    end,
-    add_unique_front(Rest, NList).
-
--spec field(map(), binary(), term()) -> term().
-field(Map, Key, Default) when is_map(Map), is_binary(Key) ->
-    maps:get(Key, Map, Default).
-
--spec to_binary(binary() | list() | atom() | any()) -> binary().
-to_binary(Value) when is_binary(Value) ->
-    Value;
-to_binary(Value) when is_list(Value) ->
-    list_to_binary(Value);
-to_binary(Value) when is_atom(Value) ->
-    atom_to_binary(Value, utf8);
-to_binary(Value) ->
-    iolist_to_binary(io_lib:format("~p", [Value])).
-
--spec to_bool(true | false | 0 | 1 | undefined) -> boolean().
-to_bool(true) ->
-    true;
-to_bool(1) ->
-    true;
-to_bool(_) ->
-    false.

apps/docker/src/docker_deployer.erl

@@ -1,233 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 07. 5月 2025 15:47
-%%%-------------------------------------------------------------------
--module(docker_deployer).
--author("anlicheng").
--dialyzer([{nowarn_function, normalize_image/1}]).
-
-%% API
--export([deploy/4]).
-
--define(TASK_SUCCESS, <<"success">>).
--define(TASK_FAIL, <<"fail">>).
-
--type reporter() :: {stream, pos_integer()}.
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
-%{
-%   "image": "nginx:latest",
-%   "container_name": "my_nginx",
-%   "ports": ["8080:80", "443:443"],
-%   "volumes": ["/host/data:/data", "/host/log:/var/log"],
-%   "envs": ["ENV1=val1", "ENV2=val2"],
-%   "entrypoint": ["/docker-entrypoint.sh"],
-%   "command": ["nginx", "-g", "daemon off;"],
-%   "restart": "always"
-%}
--spec deploy(TaskId :: integer(), ContainerDir :: string(), Params :: map(), Reporter :: reporter()) -> ok.
-deploy(TaskId, ContainerDir, Params, Reporter) when is_integer(TaskId), is_list(ContainerDir), is_map(Params) ->
-    ContainerName = deploy_container_name(Params),
-    Image0 = deploy_image(Params),
-    report_stream_event(Reporter, TaskId, <<"info">>, <<"开始部署容器:"/utf8, ContainerName/binary>>),
-    try
-        ok = ensure_container_absent(Reporter, TaskId, ContainerName),
-        {ok, Image} = ensure_image_ready(Reporter, TaskId, Image0),
-        {ok, ContainerId} = create_container_and_config(Reporter, TaskId, ContainerDir, Params),
-        ShortContainerId = short_container_id(ContainerId),
-        report_stream_event(Reporter, TaskId, <<"container_id">>, ContainerId),
-        report_stream_event(Reporter, TaskId, <<"info">>, <<"容器创建成功: "/utf8, ShortContainerId/binary>>),
-        report_stream_event(Reporter, TaskId, <<"info">>, <<"任务完成"/utf8>>),
-        write_task_summary(TaskId, <<"success">>, ContainerName, Image, ContainerId),
-        close_task(Reporter, TaskId, ?TASK_SUCCESS)
-    catch
-        throw:{deploy_error, Reason} ->
-            report_stream_event(Reporter, TaskId, <<"error">>, Reason),
-            report_stream_event(Reporter, TaskId, <<"error">>, <<"任务失败"/utf8>>),
-            write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined),
-            write_task_failure_reason(TaskId, Reason),
-            close_task(Reporter, TaskId, ?TASK_FAIL);
-        Class:Reason:Stacktrace ->
-            Error = iolist_to_binary(io_lib:format("deploy crashed: ~p:~p ~p", [Class, Reason, Stacktrace])),
-            report_stream_event(Reporter, TaskId, <<"error">>, Error),
-            report_stream_event(Reporter, TaskId, <<"error">>, <<"任务失败"/utf8>>),
-            write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined),
-            write_task_failure_reason(TaskId, Error),
-            close_task(Reporter, TaskId, ?TASK_FAIL)
-    end.
-
--spec normalize_image(binary()) -> binary().
-normalize_image(Image) when is_binary(Image) ->
-    Parts = binary:split(Image, <<"/">>, [global]),
-    {PrefixParts, [Last]} = lists:split(length(Parts) - 1, Parts),
-    NormalizedLast = case binary:split(Last, <<":">>, [global]) of
-                         [_Name] -> <<Last/binary, ":latest">>;
-                         [_Name, _Tag] -> Last
-                     end,
-    iolist_to_binary(lists:join(<<"/">>, PrefixParts ++ [NormalizedLast])).
-
--spec report_stream_event(reporter(), TaskId :: integer(), Level :: binary(), Msg :: binary()) -> ok.
-report_stream_event({stream, StreamId}, _TaskId, Level, Msg) when is_integer(StreamId), is_binary(Level), is_binary(Msg) ->
-    efka_iot_client:send_stream(StreamId, {data, encode_stream_event(Level, Msg)}).
-
--spec close_task(reporter(), integer(), binary()) -> ok.
-close_task({stream, StreamId}, _TaskId, Reason) when is_integer(StreamId), is_binary(Reason) ->
-    ok = efka_iot_client:send_stream(StreamId, {data, encode_close_event(Reason)}),
-    efka_iot_client:send_stream(StreamId, fin).
-
--spec encode_stream_event(binary(), binary()) -> binary().
-encode_stream_event(<<"container_id">>, ContainerId) ->
-    iolist_to_binary(json:encode(#{
-        <<"type">> => <<"container_id">>,
-        <<"container_id">> => ContainerId
-    }));
-encode_stream_event(Type, Msg) ->
-    iolist_to_binary(json:encode(#{
-        <<"type">> => Type,
-        <<"message">> => Msg
-    })).
-
--spec encode_close_event(binary()) -> binary().
-encode_close_event(Reason) ->
-    iolist_to_binary(json:encode(#{
-        <<"type">> => <<"close">>,
-        <<"reason">> => Reason
-    })).
-
--spec write_task_summary(integer(), binary(), binary(), binary(), undefined | binary()) -> ok.
-write_task_summary(TaskId, Status, ContainerName, Image, ContainerId)
-  when is_integer(TaskId), is_binary(Status), is_binary(ContainerName), is_binary(Image) ->
-    Fields0 = [
-        <<"type=deploy_summary">>,
-        <<"task_id=">>, integer_to_binary(TaskId),
-        <<" status=">>, Status,
-        <<" container_name=">>, ContainerName,
-        <<" image=">>, Image
-    ],
-    Fields = case ContainerId of
-                 undefined ->
-                     Fields0;
-                 ContainerId0 when is_binary(ContainerId0) ->
-                     Fields0 ++ [<<" container_id=">>, short_container_id(ContainerId0)]
-             end,
-    efka_logger:write(iolist_to_binary(Fields)).
-
--spec write_task_failure_reason(integer(), binary()) -> ok.
-write_task_failure_reason(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) ->
-    Info = iolist_to_binary([
-        <<"type=deploy_failure_reason task_id=">>,
-        integer_to_binary(TaskId),
-        <<" reason=">>,
-        Reason
-    ]),
-    efka_logger:write(Info).
-
--spec ensure_container_absent(reporter(), TaskId :: integer(), ContainerName :: binary()) -> ok.
-ensure_container_absent(Reporter, TaskId, ContainerName) when is_integer(TaskId), is_binary(ContainerName) ->
-    report_stream_event(Reporter, TaskId, <<"info">>, <<"开始创建容器: "/utf8, ContainerName/binary>>),
-    ok.
-
--spec ensure_image_ready(reporter(), TaskId :: integer(), Image0 :: binary()) -> {ok, binary()}.
-ensure_image_ready(Reporter, TaskId, Image0) when is_integer(TaskId), is_binary(Image0) ->
-    Image = normalize_image(Image0),
-    report_stream_event(Reporter, TaskId, <<"info">>, <<"使用镜像:"/utf8, Image/binary>>),
-    case docker_commands:check_image_exist(Image) of
-        true ->
-            report_stream_event(Reporter, TaskId, <<"info">>, <<"本地镜像已存在，跳过拉取:"/utf8, Image/binary>>),
-            {ok, Image};
-        false ->
-            report_stream_event(Reporter, TaskId, <<"info">>, <<"开始拉取镜像:"/utf8, Image/binary>>),
-            case docker_commands:pull_image(Image) of
-                {ok, Ref, Pid, MRef} ->
-                    await_pull_image(Reporter, TaskId, Ref, Pid, MRef),
-                    {ok, Image}
-            end
-    end.
-
--spec await_pull_image(reporter(), TaskId :: integer(), Ref :: reference(), Pid :: pid(), MRef :: reference()) -> ok.
-await_pull_image(Reporter, TaskId, Ref, Pid, MRef) ->
-    receive
-        {docker_client, Ref, {response, _Status, _Headers}} ->
-            await_pull_image(Reporter, TaskId, Ref, Pid, MRef);
-        {docker_client, Ref, {data, Data}} ->
-            report_stream_event(Reporter, TaskId, <<"info">>, Data),
-            await_pull_image(Reporter, TaskId, Ref, Pid, MRef);
-        {docker_client, Ref, done} ->
-            erlang:demonitor(MRef, [flush]),
-            ok;
-        {docker_client, Ref, {error, Reason}} ->
-            erlang:demonitor(MRef, [flush]),
-            report_stream_event(Reporter, TaskId, <<"error">>, Reason),
-            throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>});
-        {'DOWN', MRef, process, Pid, Reason0} ->
-            Reason = iolist_to_binary(io_lib:format("~p", [Reason0])),
-            throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>})
-    end.
-
--spec create_container_and_config(reporter(), TaskId :: integer(), ContainerDir :: string(), Params :: map()) ->
-    {ok, binary()}.
-create_container_and_config(Reporter, TaskId, ContainerDir, Params)
-  when is_integer(TaskId), is_list(ContainerDir), is_map(Params) ->
-    case docker_commands:create_container(ContainerDir, Params) of
-        {ok, ContainerId} ->
-            ok = create_config_file(Reporter, TaskId, ContainerDir),
-            {ok, ContainerId};
-        {error, Reason} when is_binary(Reason) ->
-            throw({deploy_error, format_create_container_error(Reason)});
-        {error, Reason} ->
-            Error = iolist_to_binary(io_lib:format("容器创建失败: ~p", [Reason])),
-            throw({deploy_error, Error})
-    end.
-
--spec create_config_file(reporter(), TaskId :: integer(), ContainerDir :: string()) -> ok.
-create_config_file(Reporter, TaskId, ContainerDir) when is_integer(TaskId), is_list(ContainerDir) ->
-    ConfigFile = docker_helper:get_config_file(ContainerDir),
-    case file:open(ConfigFile, [write, exclusive]) of
-        {ok, FD} ->
-            ok = file:write(FD, <<>>),
-            file:close(FD),
-            ok;
-        {error, Reason} ->
-            ReasonBin = list_to_binary(io_lib:format("~p", [Reason])),
-            report_stream_event(Reporter, TaskId, <<"notice">>, <<"创建配置文件失败: "/utf8, ReasonBin/binary>>),
-            ok
-    end.
-
--spec format_create_container_error(binary()) -> binary().
-format_create_container_error(Reason) when is_binary(Reason) ->
-    case is_container_already_exists_error(Reason) of
-        true ->
-            <<"本地容器已经存在"/utf8>>;
-        false ->
-            <<"容器创建失败: "/utf8, Reason/binary>>
-    end.
-
--spec is_container_already_exists_error(binary()) -> boolean().
-is_container_already_exists_error(Reason) when is_binary(Reason) ->
-    binary:match(Reason, <<"is already in use by container">>) =/= nomatch orelse
-        binary:match(Reason, <<"Conflict. The container name ">>) =/= nomatch.
-
--spec short_container_id(binary()) -> binary().
-short_container_id(ContainerId) when is_binary(ContainerId), byte_size(ContainerId) >= 12 ->
-    binary:part(ContainerId, 0, 12);
-short_container_id(ContainerId) when is_binary(ContainerId) ->
-    ContainerId.
-
--spec deploy_container_name(map()) -> binary().
-deploy_container_name(#{<<"container_name">> := ContainerName}) when is_binary(ContainerName) ->
-    ContainerName;
-deploy_container_name(_) ->
-    throw({deploy_error, <<"invalid deploy params: container_name missing">>}).
-
--spec deploy_image(map()) -> binary().
-deploy_image(#{<<"create">> := #{<<"config">> := #{<<"image">> := Image}}}) when is_binary(Image) ->
-    Image;
-deploy_image(_) ->
-    throw({deploy_error, <<"invalid deploy params: image missing">>}).

apps/docker/src/docker_helper.erl

@@ -1,102 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 17. 9月 2025 14:50
-%%%-------------------------------------------------------------------
--module(docker_helper).
--author("anlicheng").
-
-%% API
--export([root_dir/0]).
--export([ensure_container_dir/2, ensure_container_dir/3, get_container_dir/2, get_config_file/1]).
--export([update_container_config/2]).
-
--spec root_dir() -> {ok, string()} | undefined.
-root_dir() ->
-    case application:get_env(docker, root_dir) of
-        {ok, RootDir} ->
-            {ok, RootDir};
-        undefined ->
-            application:get_env(efka, root_dir)
-    end.
-
--spec ensure_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()}.
-ensure_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
-    ensure_container_dir(RootDir, ContainerName, <<>>).
-
--spec ensure_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) ->
-    {ok, ServerRootDir :: string()}.
-ensure_container_dir(RootDir, ContainerName, ContainerDir) when is_list(RootDir), is_binary(ContainerName), is_binary(ContainerDir) ->
-    PointerDir = default_container_dir(RootDir, ContainerName),
-    PointerFile = container_dir_pointer_file(PointerDir),
-    ActualContainerDir = resolve_container_dir(RootDir, ContainerName, ContainerDir),
-    ok = filelib:ensure_dir(PointerDir),
-    ok = filelib:ensure_dir(ActualContainerDir),
-    ok = file:write_file(PointerFile, unicode:characters_to_binary(ActualContainerDir), [write]),
-    {ok, ActualContainerDir}.
-
--spec get_config_file(ContainerDir :: string()) -> ConfigFile :: string().
-get_config_file(ContainerDir) when is_list(ContainerDir) ->
-    %% 根目录
-    ContainerDir ++ "service.conf".
-
--spec get_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()} | error.
-get_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
-    ContainerRootDir = default_container_dir(RootDir, ContainerName),
-    PointerFile = container_dir_pointer_file(ContainerRootDir),
-    ResolvedContainerDir = case file:read_file(PointerFile) of
-        {ok, ActualContainerDirBin} ->
-            normalize_container_dir(binary_to_list(ActualContainerDirBin));
-        {error, _} ->
-            ContainerRootDir
-    end,
-    case filelib:is_dir(ResolvedContainerDir) of
-        true ->
-            {ok, ResolvedContainerDir};
-        false ->
-            error
-    end.
-
--spec update_container_config(binary(), binary()) -> ok | {error, binary()}.
-update_container_config(ContainerName, Config) when is_binary(ContainerName), is_binary(Config) ->
-    {ok, RootDir} = root_dir(),
-    case get_container_dir(RootDir, ContainerName) of
-        {ok, ContainerDir} ->
-            ConfigFile = get_config_file(ContainerDir),
-            case file:write_file(ConfigFile, Config, [write, binary]) of
-                ok ->
-                    logger:warning("[efka_iot_client] write config file: ~p success", [ConfigFile]),
-                    ok;
-                {error, Reason} ->
-                    logger:warning("[efka_iot_client] write config file: ~p, get error: ~p", [ConfigFile, Reason]),
-                    {error, <<"write config failed">>}
-            end;
-        error ->
-            {error, <<"error">>}
-    end.
-
--spec default_container_dir(RootDir :: string(), ContainerName :: binary()) -> string().
-default_container_dir(RootDir, ContainerName) ->
-    normalize_container_dir(RootDir ++ "/" ++ binary_to_list(ContainerName)).
-
--spec container_dir_pointer_file(ContainerDir :: string()) -> string().
-container_dir_pointer_file(ContainerDir) ->
-    ContainerDir ++ ".container_dir".
-
--spec resolve_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) -> string().
-resolve_container_dir(RootDir, ContainerName, <<>>) ->
-    default_container_dir(RootDir, ContainerName);
-resolve_container_dir(_RootDir, _ContainerName, ContainerDir) ->
-    normalize_container_dir(binary_to_list(ContainerDir)).
-
--spec normalize_container_dir(ContainerDir :: string()) -> string().
-normalize_container_dir(ContainerDir) ->
-    case lists:last(ContainerDir) of
-        $/ ->
-            ContainerDir;
-        _ ->
-            ContainerDir ++ "/"
-    end.

apps/docker/src/docker_sup.erl

@@ -1,24 +0,0 @@
-%%%-------------------------------------------------------------------
-%% @doc Docker top level supervisor.
-%% @end
-%%%-------------------------------------------------------------------
-
--module(docker_sup).
-
--behaviour(supervisor).
-
--export([start_link/0]).
--export([init/1]).
-
--define(SERVER, ?MODULE).
-
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
-start_link() ->
-    supervisor:start_link({local, ?SERVER}, ?MODULE, []).
-
--spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
-init([]) ->
-    SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
-    ChildSpecs = [],
-
-    {ok, {SupFlags, ChildSpecs}}.

apps/docker/test/efka_docker_container_builder_tests.erl

@@ -1,35 +0,0 @@
--module(efka_docker_container_builder_tests).
-
--include_lib("eunit/include/eunit.hrl").
-
-port_bindings_are_rendered_for_docker_create_test() ->
-    Name = <<"nginx">>,
-    ContainerDir = "/tmp/efka-nginx/",
-    Create = #{
-        <<"config">> => #{
-            <<"image">> => <<"nginx:latest">>,
-            <<"exposed_ports">> => [
-                #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-                #{<<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
-            ]
-        },
-        <<"host_config">> => #{
-            <<"port_bindings">> => [
-                #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-                #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
-            ]
-        }
-    },
-    Options = docker_container_builder:build_options(Name, ContainerDir, Create),
-    ?assertMatch(#{
-        <<"ExposedPorts">> := #{
-            <<"80/tcp">> := #{},
-            <<"443/tcp">> := #{}
-        },
-        <<"HostConfig">> := #{
-            <<"PortBindings">> := #{
-                <<"80/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"8080">>}],
-                <<"443/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"443">>}]
-            }
-        }
-    }, Options).

apps/efka/include/efka.hrl

@@ -0,0 +1,54 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%% 扩展部分, 1: 支持基于topic的pub/sub机制; 2. 基于target的单点通讯和广播
+%%% @end
+%%% Created : 21. 4月 2025 17:28
+%%%-------------------------------------------------------------------
+-author("anlicheng").
+
+%% efka主动发起的消息体类型, 消息大类
+-define(PACKET_REQUEST, 16#01).
+-define(PACKET_RESPONSE, 16#02).
+
+%% 服务器基于pub/sub的消息, 消息大类
+-define(PACKET_PUB, 16#03).
+
+%% push调用不需要返回, 消息大类
+-define(PACKET_COMMAND, 16#04).
+
+%% 服务器端推送消息
+-define(PACKET_ASYNC_CALL, 16#05).
+-define(PACKET_ASYNC_CALL_REPLY, 16#06).
+
+%% ping包，客户端主动发起
+-define(PACKET_PING, 16#FF).
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%% 二级分类定义
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+%% 主机端上报数据类型标识
+-define(METHOD_AUTH, 16#01).
+-define(METHOD_DATA, 16#02).
+-define(METHOD_PING, 16#03).
+-define(METHOD_INFORM, 16#04).
+-define(METHOD_EVENT, 16#05).
+-define(METHOD_PHASE, 16#06).
+-define(METHOD_REQUEST_SERVICE_CONFIG, 16#07).
+
+%%%% 命令类型子分类, 不需要返回值
+
+%% 授权
+-define(COMMAND_AUTH, 16#08).
+
+%%%% 主动推送的消息类型子分类, 需要返回值
+
+-define(PUSH_DEPLOY, 16#01).
+-define(PUSH_START_SERVICE, 16#02).
+-define(PUSH_STOP_SERVICE, 16#03).
+
+-define(PUSH_SERVICE_CONFIG, 16#04).
+-define(PUSH_INVOKE, 16#05).
+-define(PUSH_TASK_LOG, 16#06).

apps/efka/include/efka_service.hrl

@@ -0,0 +1,18 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 26. 8月 2025 14:42
+%%%-------------------------------------------------------------------
+-author("anlicheng").
+
+%% 服务注册
+-define(PACKET_REQUEST, 16#01).
+%% 消息响应
+-define(PACKET_RESPONSE, 16#02).
+%% 上传数据
+-define(PACKET_PUSH, 16#03).
+
+-define(PACKET_PUB, 16#04).

apps/efka/include/efka_tables.hrl

@@ -4,21 +4,37 @@
 %%% @doc
 %%%
 %%% @end
-%%% Created : 29. 9月 2025 15:27
+%%% Created : 30. 4月 2025 11:16
 %%%-------------------------------------------------------------------
 -author("anlicheng").
 
--define(SERVICE_STOPPED, 0).
--define(SERVICE_RUNNING, 1).
-
 %% 用来保存微服务
 -record(service, {
     service_id :: binary(),
-    container_name :: binary(),
+    tar_url :: binary(),
-    %% 配置信息, 微服务启动的时候自己注册的信息
+    %% 工作目录
-    meta_data = #{} :: map(),
+    root_dir :: string(),
+    %% 配置信息
+    config_json :: binary(),
     %% 状态： 0: 停止, 1: 运行中
-    status = 0,
+    status = 0
-    create_ts  = 0 :: integer(),
+}).
-    update_ts  = 0 :: integer()
+
+%% 数据缓存
+-record(cache, {
+    id = 0 :: integer(),
+    method :: integer(),
+    data :: binary()
+}).
+
+%% 数据缓存
+-record(task_log, {
+    task_id = 0 :: integer(),
+    logs = []:: list()
+}).
+
+%% id生成器
+-record(id_generator, {
+    id,
+    value = 1
 }).

apps/efka/include/message.hrl

@@ -1,25 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2026, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 07. 7月 2026 17:09
-%%%-------------------------------------------------------------------
--author("anlicheng").
-
-%%--------------------------------------------------------------------
-%% Wire classes
-%%--------------------------------------------------------------------
--define(CLASS_REQUEST, 1).
--define(CLASS_RESPONSE, 2).
--define(CLASS_COMMAND, 3).
--define(CLASS_COMMAND_RESPONSE, 4).
--define(CLASS_MESSAGE, 5).
--define(CLASS_STREAM, 6).
-
-%%--------------------------------------------------------------------
-%% Stream targets
-%%--------------------------------------------------------------------
--define(STREAM_TARGET_MANAGER, 1).
--define(STREAM_TARGET_CONTAINER_DEPLOY, 2).

apps/efka/include/message_pb.hrl

@@ -1,224 +1,136 @@
 %% -*- coding: utf-8 -*-
 %% Automatically generated, do not edit
-%% Generated by gpb_compile version 4.21.7
+%% Generated by gpb_compile version 4.21.1
 
 -ifndef(message_pb).
 -define(message_pb, true).
 
--define(message_pb_gpb_version, "4.21.7").
+-define(message_pb_gpb_version, "4.21.1").
 
 
--ifndef('REQUEST.AUTHREQUEST_PB_H').
+-ifndef('AUTH_REQUEST_PB_H').
--define('REQUEST.AUTHREQUEST_PB_H', true).
+-define('AUTH_REQUEST_PB_H', true).
--record('Request.AuthRequest',
+-record(auth_request,
-        {uuid = <<>>            :: iodata() | undefined, % = 1, optional
+        {uuid = <<>>            :: unicode:chardata() | undefined, % = 1, optional
-         token = <<>>           :: iodata() | undefined, % = 2, optional
+         username = <<>>        :: unicode:chardata() | undefined, % = 2, optional
-         timestamp = 0          :: non_neg_integer() | undefined % = 3, optional, 64 bits
+         salt = <<>>            :: unicode:chardata() | undefined, % = 4, optional
+         token = <<>>           :: unicode:chardata() | undefined, % = 5, optional
+         timestamp = 0          :: non_neg_integer() | undefined % = 6, optional, 32 bits
         }).
 -endif.
 
--ifndef('REQUEST_PB_H').
+-ifndef('AUTH_REPLY_PB_H').
--define('REQUEST_PB_H', true).
+-define('AUTH_REPLY_PB_H', true).
--record('Request',
+-record(auth_reply,
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         body                   :: {auth_request, message_pb:'Request.AuthRequest'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('RESPONSE.ERROR_PB_H').
--define('RESPONSE.ERROR_PB_H', true).
--record('Response.Error',
         {code = 0               :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         reason = <<>>          :: iodata() | undefined % = 2, optional
+         message = <<>>         :: unicode:chardata() | undefined % = 2, optional
         }).
 -endif.
 
--ifndef('RESPONSE.AUTHRESPONSE_PB_H').
+-ifndef('PUB_PB_H').
--define('RESPONSE.AUTHRESPONSE_PB_H', true).
+-define('PUB_PB_H', true).
--record('Response.AuthResponse',
+-record(pub,
-        {
+        {topic = <<>>           :: unicode:chardata() | undefined, % = 1, optional
+         content = <<>>         :: iodata() | undefined % = 2, optional
         }).
 -endif.
 
--ifndef('RESPONSE_PB_H').
+-ifndef('ASYNC_CALL_REPLY_PB_H').
--define('RESPONSE_PB_H', true).
+-define('ASYNC_CALL_REPLY_PB_H', true).
--record('Response',
+-record(async_call_reply,
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         body                   :: {auth_response, message_pb:'Response.AuthResponse'()} | {error, message_pb:'Response.Error'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER_PB_H').
--define('COMMAND.CONTAINER_PB_H', true).
--record('Command.Container',
-        {action                 :: {list, message_pb:'Command.Container.ContainerList'()} | {start, message_pb:'Command.Container.ContainerStart'()} | {stop, message_pb:'Command.Container.ContainerStop'()} | {kill, message_pb:'Command.Container.ContainerKill'()} | {remove, message_pb:'Command.Container.ContainerRemove'()} | {config, message_pb:'Command.Container.ContainerConfig'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERCONFIG_PB_H').
--define('COMMAND.CONTAINER.CONTAINERCONFIG_PB_H', true).
--record('Command.Container.ContainerConfig',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         config = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERREMOVE_PB_H').
--define('COMMAND.CONTAINER.CONTAINERREMOVE_PB_H', true).
--record('Command.Container.ContainerRemove',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         force = false          :: boolean() | 0 | 1 | undefined, % = 2, optional
-         remove_volumes = false :: boolean() | 0 | 1 | undefined % = 3, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERKILL_PB_H').
--define('COMMAND.CONTAINER.CONTAINERKILL_PB_H', true).
--record('Command.Container.ContainerKill',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         signal = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERSTOP_PB_H').
--define('COMMAND.CONTAINER.CONTAINERSTOP_PB_H', true).
--record('Command.Container.ContainerStop',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         timeout_seconds = 0    :: non_neg_integer() | undefined % = 2, optional, 32 bits
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERSTART_PB_H').
--define('COMMAND.CONTAINER.CONTAINERSTART_PB_H', true).
--record('Command.Container.ContainerStart',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERLIST_PB_H').
--define('COMMAND.CONTAINER.CONTAINERLIST_PB_H', true).
--record('Command.Container.ContainerList',
-        {
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERTARGET_PB_H').
--define('COMMAND.CONTAINER.CONTAINERTARGET_PB_H', true).
--record('Command.Container.ContainerTarget',
-        {name = <<>>            :: iodata() | undefined, % = 1, optional
-         id = <<>>              :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('COMMAND_PB_H').
--define('COMMAND_PB_H', true).
--record('Command',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         body                   :: {container, message_pb:'Command.Container'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('COMMANDRESPONSE.ERROR_PB_H').
--define('COMMANDRESPONSE.ERROR_PB_H', true).
--record('CommandResponse.Error',
         {code = 0               :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         reason = <<>>          :: iodata() | undefined % = 2, optional
+         result = <<>>          :: unicode:chardata() | undefined, % = 2, optional
+         message = <<>>         :: unicode:chardata() | undefined % = 3, optional
         }).
 -endif.
 
--ifndef('COMMANDRESPONSE_PB_H').
+-ifndef('DEPLOY_PB_H').
--define('COMMANDRESPONSE_PB_H', true).
+-define('DEPLOY_PB_H', true).
--record('CommandResponse',
+-record(deploy,
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
+        {task_id = 0            :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         body                   :: {result, iodata()} | {error, message_pb:'CommandResponse.Error'()} | undefined % oneof
+         service_id = <<>>      :: unicode:chardata() | undefined, % = 2, optional
+         tar_url = <<>>         :: unicode:chardata() | undefined % = 3, optional
         }).
 -endif.
 
--ifndef('MESSAGE.PING_PB_H').
+-ifndef('FETCH_TASK_LOG_PB_H').
--define('MESSAGE.PING_PB_H', true).
+-define('FETCH_TASK_LOG_PB_H', true).
--record('Message.Ping',
+-record(fetch_task_log,
-        {
+        {task_id = 0            :: non_neg_integer() | undefined % = 1, optional, 32 bits
         }).
 -endif.
 
--ifndef('MESSAGE.PONG_PB_H').
+-ifndef('INVOKE_PB_H').
--define('MESSAGE.PONG_PB_H', true).
+-define('INVOKE_PB_H', true).
--record('Message.Pong',
+-record(invoke,
-        {
+        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
+         payload = <<>>         :: iodata() | undefined, % = 2, optional
+         timeout = 0            :: non_neg_integer() | undefined % = 3, optional, 32 bits
         }).
 -endif.
 
--ifndef('MESSAGE.PUB_PB_H').
+-ifndef('PUSH_SERVICE_CONFIG_PB_H').
--define('MESSAGE.PUB_PB_H', true).
+-define('PUSH_SERVICE_CONFIG_PB_H', true).
--record('Message.Pub',
+-record(push_service_config,
-        {topic = <<>>           :: iodata() | undefined, % = 1, optional
+        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-         qos = 0                :: non_neg_integer() | undefined, % = 2, optional, 32 bits
+         config_json = <<>>     :: unicode:chardata() | undefined, % = 2, optional
-         content = <<>>         :: iodata() | undefined % = 3, optional
+         timeout = 0            :: non_neg_integer() | undefined % = 3, optional, 32 bits
         }).
 -endif.
 
--ifndef('MESSAGE.METRICDATA_PB_H').
+-ifndef('DATA_PB_H').
--define('MESSAGE.METRICDATA_PB_H', true).
+-define('DATA_PB_H', true).
--record('Message.MetricData',
+-record(data,
-        {route_key = <<>>       :: iodata() | undefined, % = 1, optional
+        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-         metric = <<>>          :: iodata() | undefined % = 2, optional
+         device_uuid = <<>>     :: unicode:chardata() | undefined, % = 2, optional
+         route_key = <<>>       :: unicode:chardata() | undefined, % = 3, optional
+         metric = <<>>          :: iodata() | undefined % = 4, optional
         }).
 -endif.
 
--ifndef('MESSAGE_PB_H').
+-ifndef('PING_PB_H').
--define('MESSAGE_PB_H', true).
+-define('PING_PB_H', true).
--record('Message',
+-record(ping,
-        {body                   :: {ping, message_pb:'Message.Ping'()} | {pong, message_pb:'Message.Pong'()} | {pub, message_pb:'Message.Pub'()} | {metric_data, message_pb:'Message.MetricData'()} | undefined % oneof
+        {adcode = <<>>          :: unicode:chardata() | undefined, % = 1, optional
+         boot_time = 0          :: non_neg_integer() | undefined, % = 2, optional, 32 bits
+         province = <<>>        :: unicode:chardata() | undefined, % = 3, optional
+         city = <<>>            :: unicode:chardata() | undefined, % = 4, optional
+         efka_version = <<>>    :: unicode:chardata() | undefined, % = 5, optional
+         kernel_arch = <<>>     :: unicode:chardata() | undefined, % = 6, optional
+         ips = []               :: [unicode:chardata()] | undefined, % = 7, repeated
+         cpu_core = 0           :: non_neg_integer() | undefined, % = 8, optional, 32 bits
+         cpu_load = 0           :: non_neg_integer() | undefined, % = 9, optional, 32 bits
+         cpu_temperature = 0.0  :: float() | integer() | infinity | '-infinity' | nan | undefined, % = 10, optional
+         disk = []              :: [integer()] | undefined, % = 11, repeated, 32 bits
+         memory = []            :: [integer()] | undefined, % = 12, repeated, 32 bits
+         interfaces = <<>>      :: unicode:chardata() | undefined % = 13, optional
         }).
 -endif.
 
--ifndef('STREAM.OPEN_PB_H').
+-ifndef('SERVICE_INFORM_PB_H').
--define('STREAM.OPEN_PB_H', true).
+-define('SERVICE_INFORM_PB_H', true).
--record('Stream.Open',
+-record(service_inform,
-        {target = 0             :: non_neg_integer() | undefined % = 1, optional, 32 bits
+        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
+         props = <<>>           :: unicode:chardata() | undefined, % = 2, optional
+         status = 0             :: non_neg_integer() | undefined, % = 3, optional, 32 bits
+         timestamp = 0          :: non_neg_integer() | undefined % = 4, optional, 32 bits
         }).
 -endif.
 
--ifndef('STREAM.OPENED_PB_H').
+-ifndef('EVENT_PB_H').
--define('STREAM.OPENED_PB_H', true).
+-define('EVENT_PB_H', true).
--record('Stream.Opened',
+-record(event,
-        {
+        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
+         event_type = 0         :: non_neg_integer() | undefined, % = 2, optional, 32 bits
+         params = <<>>          :: unicode:chardata() | undefined % = 3, optional
         }).
 -endif.
 
--ifndef('STREAM.OPENERROR_PB_H').
+-ifndef('ALARM_PB_H').
--define('STREAM.OPENERROR_PB_H', true).
+-define('ALARM_PB_H', true).
--record('Stream.OpenError',
+-record(alarm,
-        {reason = <<>>          :: iodata() | undefined % = 1, optional
+        {service_id = <<>>      :: unicode:chardata() | undefined, % = 1, optional
-        }).
+         params = <<>>          :: unicode:chardata() | undefined % = 2, optional
--endif.
-
--ifndef('STREAM.DATA_PB_H').
--define('STREAM.DATA_PB_H', true).
--record('Stream.Data',
-        {bytes = <<>>           :: iodata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('STREAM.FIN_PB_H').
--define('STREAM.FIN_PB_H', true).
--record('Stream.Fin',
-        {
-        }).
--endif.
-
--ifndef('STREAM.RESET_PB_H').
--define('STREAM.RESET_PB_H', true).
--record('Stream.Reset',
-        {reason = <<>>          :: iodata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('STREAM_PB_H').
--define('STREAM_PB_H', true).
--record('Stream',
-        {stream_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         payload                :: {open, message_pb:'Stream.Open'()} | {opened, message_pb:'Stream.Opened'()} | {open_error, message_pb:'Stream.OpenError'()} | {data, message_pb:'Stream.Data'()} | {fin, message_pb:'Stream.Fin'()} | {reset, message_pb:'Stream.Reset'()} | undefined % oneof
         }).
 -endif.
 

apps/efka/include/service_pb.hrl

@@ -1,72 +0,0 @@
-%% -*- coding: utf-8 -*-
-%% Automatically generated, do not edit
-%% Generated by gpb_compile version 4.21.7
-
--ifndef(service_pb).
--define(service_pb, true).
-
--define(service_pb_gpb_version, "4.21.7").
-
-
--ifndef('SERVICEREQUEST.REGISTER_PB_H').
--define('SERVICEREQUEST.REGISTER_PB_H', true).
--record('ServiceRequest.Register',
-        {service_id = <<>>      :: unicode:chardata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('SERVICEREQUEST.SUBSCRIBE_PB_H').
--define('SERVICEREQUEST.SUBSCRIBE_PB_H', true).
--record('ServiceRequest.Subscribe',
-        {topic = <<>>           :: unicode:chardata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('SERVICEREQUEST_PB_H').
--define('SERVICEREQUEST_PB_H', true).
--record('ServiceRequest',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         request                :: {register, service_pb:'ServiceRequest.Register'()} | {subscribe, service_pb:'ServiceRequest.Subscribe'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('SERVICEREPLY.ERROR_PB_H').
--define('SERVICEREPLY.ERROR_PB_H', true).
--record('ServiceReply.Error',
-        {code = 0               :: integer() | undefined, % = 1, optional, 32 bits
-         message = <<>>         :: unicode:chardata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('SERVICEREPLY_PB_H').
--define('SERVICEREPLY_PB_H', true).
--record('ServiceReply',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         reply                  :: {result, iodata()} | {error, service_pb:'ServiceReply.Error'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('SERVICECAST.METRICDATA_PB_H').
--define('SERVICECAST.METRICDATA_PB_H', true).
--record('ServiceCast.MetricData',
-        {route_key = <<>>       :: iodata() | undefined, % = 1, optional
-         metric = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('SERVICECAST.TOPICEVENT_PB_H').
--define('SERVICECAST.TOPICEVENT_PB_H', true).
--record('ServiceCast.TopicEvent',
-        {topic = <<>>           :: unicode:chardata() | undefined, % = 1, optional
-         content = <<>>         :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('SERVICECAST_PB_H').
--define('SERVICECAST_PB_H', true).
--record('ServiceCast',
-        {body                   :: {topic_event, service_pb:'ServiceCast.TopicEvent'()} | {metric_data, service_pb:'ServiceCast.MetricData'()} | undefined % oneof
-        }).
--endif.
-
--endif.

apps/efka/proto/message.proto

@@ -1,179 +0,0 @@
-syntax = "proto3";
-
-// iot <-> efka protocol payload definitions.
-//
-// The transport frame keeps the first byte as the coarse message class:
-//
-//   [CLASS_REQUEST][protobuf(Request)]
-//   [CLASS_RESPONSE][protobuf(Response)]
-//   [CLASS_COMMAND][protobuf(Command)]
-//   [CLASS_COMMAND_RESPONSE][protobuf(CommandResponse)]
-//   [CLASS_MESSAGE][protobuf(Message)]
-//   [CLASS_STREAM][protobuf(Stream)]
-//
-// HTTP proxy bytes carried by Stream.Data are transparent payload bytes.
-
-message Request {
-  uint64 packet_id = 1;
-
-  message AuthRequest {
-    bytes uuid = 1;
-    bytes token = 2;
-    uint64 timestamp = 3;
-  }
-
-  oneof body {
-    AuthRequest auth_request = 10;
-  }
-
-}
-
-message Response {
-  uint64 packet_id = 1;
-
-  message Error {
-    uint32 code = 1;
-    bytes reason = 2;
-  }
-
-  message AuthResponse {
-
-  }
-
-  oneof body {
-    AuthResponse auth_response = 10;
-    Error error = 11;
-  }
-
-}
-
-message Command {
-  uint64 packet_id = 1;
-
-  message Container {
-    message ContainerTarget {
-      bytes name = 1;
-      bytes id = 2;
-    }
-
-    message ContainerList {
-
-    }
-
-    message ContainerStart {
-      ContainerTarget target = 1;
-    }
-
-    message ContainerStop {
-      ContainerTarget target = 1;
-      uint32 timeout_seconds = 2;
-    }
-
-    message ContainerKill {
-      ContainerTarget target = 1;
-      bytes signal = 2;
-    }
-
-    message ContainerRemove {
-      ContainerTarget target = 1;
-      bool force = 2;
-      bool remove_volumes = 3;
-    }
-
-    message ContainerConfig {
-      ContainerTarget target = 1;
-      bytes config = 2;
-    }
-
-    oneof action {
-      ContainerList list = 1;
-      ContainerStart start = 3;
-      ContainerStop stop = 4;
-      ContainerKill kill = 5;
-      ContainerRemove remove = 6;
-      ContainerConfig config = 7;
-    }
-
-  }
-
-  oneof body {
-    Container container = 10;
-  }
-
-}
-
-message CommandResponse {
-  uint64 packet_id = 1;
-
-  message Error {
-    uint32 code = 1;
-    bytes reason = 2;
-  }
-
-  oneof body {
-    bytes result = 10;
-    Error error = 11;
-  }
-
-}
-
-message Message {
-  message Ping {
-  }
-
-  message Pong {
-  }
-
-  message Pub {
-    bytes topic = 1;
-    uint32 qos = 2;
-    bytes content = 3;
-  }
-
-  message MetricData {
-    bytes route_key = 1;
-    bytes metric = 2;
-  }
-
-  oneof body {
-    Ping ping = 10;
-    Pong pong = 11;
-    Pub pub = 12;
-    MetricData metric_data = 13;
-  }
-}
-
-message Stream {
-  uint32 stream_id = 1;
-
-  message Open {
-    uint32 target = 1;
-  }
-
-  message Opened {
-  }
-
-  message OpenError {
-    bytes reason = 1;
-  }
-
-  message Data {
-    bytes bytes = 1;
-  }
-
-  message Fin {
-  }
-
-  message Reset {
-    bytes reason = 1;
-  }
-
-  oneof payload {
-    Open open = 10;
-    Opened opened = 11;
-    OpenError open_error = 12;
-    Data data = 13;
-    Fin fin = 14;
-    Reset reset = 15;
-  }
-}

apps/efka/proto/service.proto

@@ -1,54 +0,0 @@
-syntax = "proto3";
-
-// 微服务通过 ws_channel 与 efka 之间的通道消息定义。
-// 该协议用于替代当前基于 JSON 的请求/响应/推送格式。
-
-message ServiceRequest {
-  uint64 packet_id = 1;
-
-  message Register {
-    string service_id = 1;
-  }
-
-  message Subscribe {
-    string topic = 1;
-  }
-
-  oneof request {
-    Register register = 10;
-    Subscribe subscribe = 11;
-  }
-}
-
-message ServiceReply {
-  uint64 packet_id = 1;
-
-  message Error {
-    int32 code = 1;
-    string message = 2;
-  }
-
-  oneof reply {
-    bytes result = 10;
-    Error error = 11;
-  }
-}
-
-message ServiceCast {
-
-  message MetricData {
-    bytes route_key = 1;
-    bytes metric = 2;
-  }
-
-  message TopicEvent {
-    string topic = 1;
-    bytes content = 2;
-  }
-
-  oneof body {
-    TopicEvent topic_event = 10;
-    MetricData metric_data = 11;
-  }
-
-}

apps/efka/rebar.config

@@ -1,23 +0,0 @@
-{erl_opts, [{i, "include"}]}.
-
-{gpb_opts, [
-    {i, "proto"},
-    {f, ["service.proto", "message.proto"]},
-    recursive,
-    {module_name_prefix, ""},
-    {module_name_suffix, "_pb"},
-    {o_erl, "src/protobuf"},
-    {o_hrl, "include"},
-    include_as_lib,
-    {strings_as_binaries, true},
-    type_specs,
-    report,
-    verbose
-]}.
-
-{provider_hooks, [
-    {pre, [
-        {compile, {protobuf, compile}},
-        {clean, {protobuf, clean}}
-    ]}
-]}.

apps/efka/src/efka.app.src

@@ -5,15 +5,22 @@
   {mod, {efka_app, []}},
   {applications,
    [
-    %sync,
+    sync,
-    docker,
+    jiffy,
+    %gpb,
+    parse_trans,
+    lager,
     cowboy,
     ranch,
     crypto,
-    cowlib,
     inets,
     ssl,
     public_key,
+
+    %erts,
+    %runtime_tools,
+    %observer,
+
     kernel,
     stdlib
    ]},

apps/efka/src/efka_app.erl

@@ -9,38 +9,14 @@
 
 -export([start/2, stop/1]).
 
--spec start(term(), term()) -> {ok, pid()} | {error, term()}.
 start(_StartType, _StartArgs) ->
     io:setopts([{encoding, unicode}]),
     %% 加速内存的回收
     erlang:system_flag(fullsweep_after, 16),
-    start_websocket_server(),
+
+    ws_server:start_server(),
 
     efka_sup:start_link().
 
--spec stop(term()) -> ok.
 stop(_State) ->
     ok.
-
-%% 微服务和efka之间通过websocket协议通讯
--spec start_websocket_server() -> ok.
-start_websocket_server() ->
-    {ok, Props} = application:get_env(efka, websocket_server),
-    Acceptors = proplists:get_value(acceptors, Props, 50),
-    MaxConnections = proplists:get_value(max_connections, Props, 10240),
-    Backlog = proplists:get_value(backlog, Props, 1024),
-    Port = proplists:get_value(port, Props),
-
-    Dispatcher = cowboy_router:compile([
-        {'_', [{"/ws", efka_service_channel, []}]}
-    ]),
-
-    TransOpts = [
-        {port, Port},
-        {num_acceptors, Acceptors},
-        {backlog, Backlog},
-        {max_connections, MaxConnections}
-    ],
-    {ok, Pid} = cowboy:start_clear(ws_listener, TransOpts, #{env => #{dispatch => Dispatcher}}),
-
-    logger:debug("[efka_app] websocket server start at: ~p, pid is: ~p", [Port, Pid]).

apps/efka/src/efka_inetd.erl

@@ -0,0 +1,212 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%% 微服务守护进程
+%%% 1. 负责微服务的下载, 版本的管理
+%%% 2. 目录管理等
+%%% @end
+%%% Created : 19. 4月 2025 14:55
+%%%-------------------------------------------------------------------
+-module(efka_inetd).
+-author("anlicheng").
+-include("efka_tables.hrl").
+-include("message_pb.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0]).
+-export([deploy/3, start_service/1, stop_service/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+-record(state, {
+    root_dir :: string(),
+    %% 建立任务到ref之间的映射, #{TaskPid => {TaskId, ServiceId}}
+    task_map = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec deploy(TaskId :: integer(), ServerId :: binary(), TarUrl :: binary()) -> ok | {error, Reason :: binary()}.
+deploy(TaskId, ServerId, TarUrl) when is_integer(TaskId), is_binary(ServerId), is_binary(TarUrl) ->
+    gen_server:call(?SERVER, {deploy, TaskId, ServerId, TarUrl}).
+
+-spec start_service(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
+start_service(ServiceId) when is_binary(ServiceId) ->
+    gen_server:call(?SERVER, {start_service, ServiceId}).
+
+-spec stop_service(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
+stop_service(ServiceId) when is_binary(ServiceId) ->
+    gen_server:call(?SERVER, {stop_service, ServiceId}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link() ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([]) ->
+    erlang:process_flag(trap_exit, true),
+    {ok, RootDir} = application:get_env(efka, root_dir),
+    {ok, #state{root_dir = RootDir}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call({deploy, TaskId, ServiceId, TarUrl}, _From, State = #state{root_dir = RootDir, task_map = TaskMap}) ->
+    %% 创建目录
+    {ok, ServiceRootDir} = ensure_dirs(RootDir, ServiceId),
+
+    ServicePid = efka_service:get_pid(ServiceId),
+    case is_pid(ServicePid) of
+        true ->
+            {reply, {error, <<"the service is running, stop first">>}, State};
+        false ->
+            case check_download_url(TarUrl) of
+                ok ->
+                    {ok, TaskPid} = efka_inetd_task:start_link(TaskId, ServiceRootDir, ServiceId, TarUrl),
+                    efka_inetd_task:deploy(TaskPid),
+                    lager:debug("[efka_inetd] start task_id: ~p, tar_url: ~p", [TaskId, TarUrl]),
+
+                    {reply, ok, State#state{task_map = maps:put(TaskPid, {TaskId, ServiceId}, TaskMap)}};
+                {error, Reason} ->
+                    lager:debug("[efka_inetd] check_download_url: ~p, get error: ~p", [TarUrl, Reason]),
+                    {reply, {error, <<"download url error">>}, State}
+            end
+    end;
+
+%% 启动服务: 当前服务如果正常运行，则不允许重启
+handle_call({start_service, ServiceId}, _From, State) ->
+    case efka_service:get_pid(ServiceId) of
+        undefined ->
+            case efka_service_sup:start_service(ServiceId) of
+                {ok, _} ->
+                    %% 更新数据库状态, 状态是为了保证下次efka重启的时候，服务能够启动
+                    ok = service_model:change_status(ServiceId, 1),
+                    {reply, ok, State};
+                {error, Reason} ->
+                    {reply, {error, Reason}, State}
+            end;
+        ServicePid when is_pid(ServicePid) ->
+            {reply, {error, <<"service is running">>}, State}
+    end;
+
+%% 停止服务, 主动停止的时候会改变服务配置的status字段
+handle_call({stop_service, ServiceId}, _From, State = #state{}) ->
+    case efka_service:get_pid(ServiceId) of
+        undefined ->
+            {reply, {error, <<"service not running">>}, State};
+        ServicePid when is_pid(ServicePid) ->
+            efka_service_sup:stop_service(ServiceId),
+            %% 主动停止的服务，需要更新数据库状态, 状态是为了保证下次efka重启的时候，不自动启动服务
+            ok = service_model:change_status(ServiceId, 0),
+            {reply, ok, State}
+    end;
+
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info({'EXIT', TaskPid, Reason}, State = #state{task_map = TaskMap}) ->
+    case maps:take(TaskPid, TaskMap) of
+        error ->
+            {noreply, State};
+        {{TaskId, ServiceId}, NTaskMap} ->
+            case Reason of
+                normal ->
+                    lager:debug("[efka_inetd] service_id: ~p, task_pid: ~p, exit normal", [ServiceId, TaskPid]),
+                    efka_inetd_task_log:flush(TaskId);
+                Error ->
+                    lager:notice("[efka_inetd] service_id: ~p, task_pid: ~p, exit with error: ~p", [ServiceId, TaskPid, Error]),
+                    efka_inetd_task_log:stash(TaskId, <<"task aborted">>),
+                    efka_inetd_task_log:flush(TaskId)
+            end,
+            {noreply, State#state{task_map = NTaskMap}}
+    end;
+
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec ensure_dirs(RootDir :: string(), ServerId :: binary()) -> {ok, ServerRootDir :: string()}.
+ensure_dirs(RootDir, ServerId) when is_list(RootDir), is_binary(ServerId) ->
+    %% 根目录
+    ServiceRootDir = RootDir ++ "/" ++ binary_to_list(ServerId) ++ "/",
+    ok = filelib:ensure_dir(ServiceRootDir),
+    {ok, ServiceRootDir}.
+
+%% 通过head请求先判定下载地址是否正确
+-spec check_download_url(Url :: string() | binary()) -> ok | {error, Reason :: term()}.
+check_download_url(Url) when is_binary(Url) ->
+    check_download_url(binary_to_list(Url));
+check_download_url(Url) when is_list(Url) ->
+    SslOpts = [
+        {ssl, [
+            % 完全禁用证书验证
+            {verify, verify_none}
+        ]}
+    ],
+    case httpc:request(head, {Url, []}, SslOpts, [{sync, true}]) of
+        {ok, {{_, 200, "OK"}, _Headers, _}} ->
+            ok;
+        {error, Reason} ->
+            {error, Reason}
+    end.

apps/efka/src/efka_inetd_task.erl

@@ -0,0 +1,241 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 07. 5月 2025 15:47
+%%%-------------------------------------------------------------------
+-module(efka_inetd_task).
+-author("anlicheng").
+-include("efka_tables.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/4]).
+-export([deploy/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+-record(state, {
+    service_root_dir :: string(),
+    task_id :: integer(),
+    service_id :: binary(),
+    tar_url :: binary()
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec deploy(Pid :: pid()) -> no_return().
+deploy(Pid) when is_pid(Pid) ->
+    gen_server:cast(Pid, deploy).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link(TaskId :: integer(), ServiceRootDir :: string(), ServiceId :: binary(), TarUrl :: binary()) ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link(TaskId, ServiceRootDir, ServiceId, TarUrl) when is_integer(TaskId), is_list(ServiceRootDir), is_binary(ServiceId), is_binary(TarUrl) ->
+    gen_server:start_link(?MODULE, [TaskId, ServiceRootDir, ServiceId, TarUrl], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([TaskId, ServiceRootDir, ServiceId, TarUrl]) ->
+    {ok, #state{task_id = TaskId, service_root_dir = ServiceRootDir, service_id = ServiceId, tar_url = TarUrl}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast(deploy, State = #state{task_id = TaskId, service_root_dir = ServiceRootDir, service_id = ServiceId, tar_url = TarUrl}) ->
+    do_deploy(TaskId, ServiceRootDir, ServiceId, TarUrl),
+    {stop, normal, State};
+handle_cast(_Request, State) ->
+    {stop, normal, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec do_deploy(TaskId :: integer(), ServiceRootDir :: string(), ServiceId :: binary(), TarUrl :: binary()) -> no_return().
+do_deploy(TaskId, ServiceRootDir, ServiceId, TarUrl) when is_integer(TaskId), is_list(ServiceRootDir), is_binary(ServiceId), is_binary(TarUrl) ->
+    case download(binary_to_list(TarUrl), ServiceRootDir) of
+        {ok, TarFile, CostTs} ->
+            Log = io_lib:format("download: ~p completed, cost time: ~p(ms)", [binary_to_list(TarUrl), CostTs]),
+            efka_inetd_task_log:stash(TaskId, list_to_binary(Log)),
+
+            %% 创建工作目录
+            WorkDir = ServiceRootDir ++ "/work_dir/",
+            case filelib:ensure_dir(WorkDir) of
+                ok ->
+                    %% 清理目录下的文件
+                    catch delete_directory(WorkDir),
+                    case tar_extract(TarFile, WorkDir) of
+                        ok ->
+                            %% 更新数据
+                            ok = service_model:insert(#service{
+                                service_id = ServiceId,
+                                tar_url = TarUrl,
+                                %% 工作目录
+                                root_dir = ServiceRootDir,
+                                config_json = <<"">>,
+                                %% 状态： 0: 停止, 1: 运行中
+                                status = 0
+                            }),
+                            efka_inetd_task_log:stash(TaskId, <<"deploy success">>);
+                        {error, Reason} ->
+                            TarLog = io_lib:format("tar decompression: ~p, error: ~p", [filename:basename(TarFile), Reason]),
+                            efka_inetd_task_log:stash(TaskId, list_to_binary(TarLog))
+                    end;
+                {error, Reason} ->
+                    DownloadLog = io_lib:format("make work_dir error: ~p", [Reason]),
+                    efka_inetd_task_log:stash(TaskId, list_to_binary(DownloadLog))
+            end;
+        {error, Reason} ->
+            DownloadLog = io_lib:format("download: ~p, error: ~p", [binary_to_list(TarUrl), Reason]),
+            efka_inetd_task_log:stash(TaskId, list_to_binary(DownloadLog))
+    end.
+
+%% 递归删除目录下的问题
+-spec delete_directory(string()) -> ok | {error, term()}.
+delete_directory(Dir) when is_list(Dir) ->
+    % 递归删除目录内容
+    case file:list_dir(Dir) of
+        {ok, Files} ->
+            lists:foreach(fun(File) ->
+                FullPath = filename:join(Dir, File),
+                case filelib:is_dir(FullPath) of
+                    true ->
+                        delete_directory(FullPath);
+                    false ->
+                        file:delete(FullPath)
+                end
+                          end, Files),
+            % 删除空目录
+            file:del_dir(Dir);
+        {error, enoent} ->
+            ok;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+%% 解压文件到指定目录
+-spec tar_extract(string(), string()) -> ok | {error, term()}.
+tar_extract(TarFile, TargetDir) when is_list(TarFile), is_list(TargetDir) ->
+    %% 判断文件的后缀名来判断, options: verbose
+    erl_tar:extract(TarFile, [compressed, {cwd, TargetDir}]).
+
+%% 下载文件
+-spec download(Url :: string(), TargetDir :: string()) ->
+    {ok, TarFile :: string(), CostTs :: integer()} | {error, Reason :: term()}.
+download(Url, TargetDir) when is_list(Url), is_list(TargetDir) ->
+    SslOpts = [
+        {ssl, [
+            % 完全禁用证书验证
+            {verify, verify_none}
+        ]}
+    ],
+
+    TargetFile = get_filename_from_url(Url),
+    FullFilename = TargetDir ++ TargetFile,
+
+    StartTs = os:timestamp(),
+    case httpc:request(get, {Url, []}, SslOpts, [{sync, false}, {stream, self}]) of
+        {ok, RequestId} ->
+            case receive_data(RequestId, FullFilename) of
+                ok ->
+                    EndTs = os:timestamp(),
+                    %% 计算操作的时间，单位为毫秒
+                    CostMs = timer:now_diff(EndTs, StartTs) div 1000,
+                    {ok, FullFilename, CostMs};
+                {error, Reason} ->
+                    %% 出错需要删除掉文件
+                    file:delete(FullFilename),
+                    {error, Reason}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+%% 处理头部信息, 解析可能的文件名
+receive_data(RequestId, FullFilename) ->
+    receive
+        {http, {RequestId, stream_start, _Headers}} ->
+            {ok, File} = file:open(FullFilename, [write, binary]),
+            receive_data0(RequestId, File);
+        {http, {RequestId, {{_, 404, Status}, _Headers, Body}}} ->
+            lager:debug("[efka_downloader] http_status: ~p, body: ~p", [Status, Body]),
+            {error, Status}
+    end.
+%% 接受文件数据
+receive_data0(RequestId, File) ->
+    receive
+        {http, {RequestId, {error, Reason}}} ->
+            ok = file:close(File),
+            {error, Reason};
+        {http, {RequestId, stream_end, _Headers}} ->
+            ok = file:close(File),
+            ok;
+        {http, {RequestId, stream, Data}} ->
+            file:write(File, Data),
+            receive_data0(RequestId, File)
+    end.
+
+-spec get_filename_from_url(Url :: string()) -> string().
+get_filename_from_url(Url) when is_list(Url) ->
+    URIMap = uri_string:parse(Url),
+    Path = maps:get(path, URIMap),
+    filename:basename(Path).

apps/efka/src/efka_inetd_task_log.erl

@@ -0,0 +1,136 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 09. 5月 2025 16:45
+%%%-------------------------------------------------------------------
+-module(efka_inetd_task_log).
+-author("anlicheng").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0]).
+-export([stash/2, flush/1, get_logs/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+-record(state, {
+    %% #{task_id => queue:new()}
+    pending_map = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec stash(TaskId :: integer(), Items :: binary() | [binary()]) -> no_return().
+stash(TaskId, Log) when is_integer(TaskId), is_binary(Log) ->
+    stash(TaskId, [Log]);
+stash(TaskId, Items) when is_integer(TaskId), is_list(Items) ->
+    {{Y, M, D}, {H, I, S}} = calendar:local_time(),
+    TimePrefix = iolist_to_binary(io_lib:format("[~b-~2..0b-~2..0b ~2..0b:~2..0b:~2..0b]", [Y, M, D, H, I, S])),
+    Log = iolist_to_binary([TimePrefix, <<" ">>, lists:join(<<" ">>, Items)]),
+    gen_server:cast(?SERVER, {stash, TaskId, Log}).
+
+-spec flush(TaskId :: integer()) -> no_return().
+flush(TaskId) when is_integer(TaskId) ->
+    gen_server:cast(?SERVER, {flush, TaskId}).
+
+-spec get_logs(TaskId :: integer()) -> {ok, Logs :: list()}.
+get_logs(TaskId) when is_integer(TaskId) ->
+    gen_server:call(?SERVER, {get_logs, TaskId}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link() ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([]) ->
+    {ok, #state{}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call({get_logs, TaskId}, _From, State = #state{pending_map = PendingMap}) ->
+    case maps:find(TaskId, PendingMap) of
+        error ->
+            Logs = task_log_model:get_logs(TaskId),
+            {reply, {ok, Logs}, State};
+        {ok, Q} ->
+            Logs = queue:to_list(Q),
+            {reply, {ok, Logs}, State}
+    end.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast({stash, TaskId, Log}, State = #state{pending_map = PendingMap}) ->
+    Q = maps:get(TaskId, PendingMap, queue:new()),
+    NQ = queue:in(Log, Q),
+    {noreply, State#state{pending_map = maps:put(TaskId, NQ, PendingMap)}};
+handle_cast({flush, TaskId}, State = #state{pending_map = PendingMap}) ->
+    case maps:take(TaskId, PendingMap) of
+        error ->
+            {noreply, State};
+        {Q, NPendingMap} ->
+            Logs = queue:to_list(Q),
+            ok = task_log_model:insert(TaskId, Logs),
+            {noreply, State#state{pending_map = NPendingMap}}
+    end.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================

apps/efka/src/efka_logger.erl

@@ -1,179 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author aresei
-%%% @copyright (C) 2023, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 07. 9月 2023 17:07
-%%%-------------------------------------------------------------------
--module(efka_logger).
--author("aresei").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/1, write/1, write_lines/1]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE).
-
--record(state, {
-    file_name :: string(),
-    date :: calendar:date(),
-    file
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec write(Data :: binary()) -> ok.
-write(Data) when is_binary(Data) ->
-    gen_server:cast(?SERVER, {write, Data}).
-
--spec write_lines(Lines :: [binary()]) -> ok.
-write_lines(Lines) when is_list(Lines) ->
-    gen_server:cast(?SERVER, {write_lines, Lines}).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link(FileName :: string()) ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link(FileName) when is_list(FileName) ->
-    gen_server:start_link({local, ?SERVER}, ?MODULE, [FileName], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([FileName]) ->
-    ensure_dir(),
-    FilePath = make_file(FileName),
-    {ok, File} = file:open(FilePath, [append, binary]),
-
-    {ok, #state{file = File, file_name = FileName, date = get_date()}}.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast({write, Data}, State = #state{file = OldFile, file_name = FileName, date = Date}) ->
-    Line = <<(time_prefix())/binary, " ", (format(Data))/binary, $\n>>,
-    case maybe_new_file(Date) of
-        true ->
-            file:close(OldFile),
-
-            FilePath = make_file(FileName),
-            {ok, File} = file:open(FilePath, [append, binary]),
-            ok = file:write(File, Line),
-            {noreply, State#state{file = File, date = get_date()}};
-        false ->
-            ok = file:write(OldFile, Line),
-            {noreply, State}
-    end;
-handle_cast({write_lines, Lines}, State = #state{file = OldFile, file_name = FileName, date = Date}) ->
-    Data = iolist_to_binary(lists:join(<<$\n>>, Lines)),
-    case maybe_new_file(Date) of
-        true ->
-            file:close(OldFile),
-
-            FilePath = make_file(FileName),
-            {ok, File} = file:open(FilePath, [append, binary]),
-            ok = file:write(File, Data),
-            {noreply, State#state{file = File, date = get_date()}};
-        false ->
-            ok = file:write(OldFile, Data),
-            {noreply, State}
-    end.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_info(_Info, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(_Reason, _State = #state{}) ->
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec format(binary() | [iodata()]) -> binary().
-format(Data) when is_binary(Data) ->
-    iolist_to_binary(Data);
-format(Items) when is_list(Items) ->
-    iolist_to_binary(lists:join(<<"\t">>, Items)).
-
--spec time_prefix() -> binary().
-time_prefix() ->
-    {{Y, M, D}, {H, I, S}} = calendar:local_time(),
-    iolist_to_binary(io_lib:format("[~b-~2..0b-~2..0b ~2..0b:~2..0b:~2..0b]", [Y, M, D, H, I, S])).
-
--spec make_file(LogFile :: string()) -> string().
-make_file(LogFile) when is_list(LogFile) ->
-    {Year, Month, Day} = erlang:date(),
-    Suffix = io_lib:format("~b~2..0b~2..0b", [Year, Month, Day]),
-    RootDir = code:root_dir() ++ "/log/",
-    lists:flatten(RootDir ++ LogFile ++ "." ++ Suffix).
-
--spec ensure_dir() -> ok | {error, term()}.
-ensure_dir() ->
-    RootDir = code:root_dir() ++ "/log/",
-    case filelib:is_dir(RootDir) of
-        true ->
-            ok;
-        false ->
-            file:make_dir(RootDir)
-    end.
-
-%% 获取日期信息
--spec get_date() -> Date :: calendar:date().
-get_date() ->
-    {Date, _} = calendar:local_time(),
-    Date.
-
-%% 通过日志判断是否需要生成新的日志文件
--spec maybe_new_file(Date :: calendar:date()) -> boolean().
-maybe_new_file({Y, M, D}) ->
-    {{Y0, M0, D0}, _} = calendar:local_time(),
-    not (Y =:= Y0 andalso M =:= M0 andalso D =:= D0).

apps/efka/src/efka_manifest.erl

@@ -0,0 +1,124 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%% 用于管理manifest.json配置文件
+%%% @end
+%%% Created : 05. 5月 2025 22:39
+%%%-------------------------------------------------------------------
+-module(efka_manifest).
+-author("anlicheng").
+
+-record(manifest, {
+    work_dir = "" :: string(),
+    id = <<"">> :: binary(),
+    exec = <<"">>:: binary(),
+    args = [],
+    health_check = <<"">>
+}).
+
+-type manifest() :: #manifest{}.
+
+-export_type([manifest/0]).
+
+%% API
+-export([new/1, startup/1]).
+
+-spec new(ServiceRootDir :: string()) -> {ok, #manifest{}} | {error, Reason :: binary()}.
+new(ServiceRootDir) when is_list(ServiceRootDir) ->
+    WorkDir = ServiceRootDir ++ "/work_dir/",
+    case file:read_file(WorkDir ++ "manifest.json") of
+        {ok, ManifestInfo} ->
+            Settings = catch jiffy:decode(ManifestInfo, [return_maps]),
+            case check_manifest(Settings) of
+                {ok, Manifest} ->
+                    {ok, Manifest#manifest{work_dir = WorkDir}};
+                {error, Reason} ->
+                    {error, Reason}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec startup(Manifest :: #manifest{}) -> {ok, Port :: port()} | {error, Reason :: binary()}.
+startup(#manifest{id = Id, work_dir = WorkDir, exec = ExecCmd0, args = Args0}) ->
+    PortSettings = [
+        {cd, WorkDir},
+        {args, [binary_to_list(A) || A <- Args0]},
+        exit_status
+    ],
+    ExecCmd = binary_to_list(ExecCmd0),
+    RealExecCmd = filename:absname_join(WorkDir, ExecCmd),
+    lager:debug("[efka_manifest] service_id: ~p, real command is: ~p", [Id, RealExecCmd]),
+    case catch erlang:open_port({spawn_executable, RealExecCmd}, PortSettings) of
+        Port when is_port(Port) ->
+            {ok, Port};
+        _Other ->
+            {error, <<"exec command startup failed">>}
+    end.
+
+%% 检查配置是否合法
+-spec check_manifest(Manifest :: map()) -> {ok, #manifest{}} | {error, Reason :: binary()}.
+check_manifest(Manifest) when is_map(Manifest) ->
+    RequiredKeys = [<<"id">>, <<"exec">>, <<"args">>, <<"health_check">>],
+    check_manifest0(RequiredKeys, Manifest, #manifest{});
+check_manifest(_Manifest) ->
+    {error, <<"invalid manifest json">>}.
+
+check_manifest0([], _Settings, Manifest) ->
+    {ok, Manifest};
+check_manifest0([<<"id">>|T], Settings, Manifest) ->
+    case maps:find(<<"id">>, Settings) of
+        error ->
+            {error, <<"miss service_id">>};
+        {ok, Id} when is_binary(Id) ->
+            check_manifest0(T, Settings, Manifest#manifest{id = Id});
+        {ok, _} ->
+            {error, <<"service_id is not string">>}
+    end;
+check_manifest0([<<"health_check">>|T], Settings, Manifest) ->
+    case maps:find(<<"health_check">>, Settings) of
+        error ->
+            {error, <<"miss health_check">>};
+        {ok, Url} when is_binary(Url) ->
+            case is_url(Url) of
+                true ->
+                    check_manifest0(T, Settings, Manifest#manifest{health_check = Url});
+                false ->
+                    {error, <<"health_check is not a invalid url">>}
+            end;
+        {ok, _} ->
+            {error, <<"health_check is not string">>}
+    end;
+check_manifest0([<<"exec">>|T], Settings, Manifest) ->
+    case maps:find(<<"exec">>, Settings) of
+        error ->
+            {error, <<"miss start">>};
+        {ok, Exec} when is_binary(Exec) ->
+            %% 不能包含空格
+            case binary:match(Exec, <<" ">>) of
+                nomatch ->
+                    check_manifest0(T, Settings, Manifest#manifest{exec = Exec});
+                _ ->
+                    {error, <<"start cmd cannot contain args">>}
+            end
+    end;
+check_manifest0([<<"args">>|T], Settings, Manifest) ->
+    case maps:find(<<"args">>, Settings) of
+        error ->
+            check_manifest0(T, Settings, Manifest#manifest{args = []});
+        %% 对参数项目不进行检查
+        {ok, Args} when is_list(Args) ->
+            check_manifest0(T, Settings, Manifest#manifest{args = Args});
+        {ok, _} ->
+            {error, <<"args must be list">>}
+    end.
+
+-spec is_url(binary()) -> boolean().
+is_url(Input) when is_binary(Input) ->
+    try
+        uri_string:parse(Input),
+        true
+    catch
+        _:_ -> false
+    end.

apps/efka/src/efka_monitor.erl

@@ -0,0 +1,25 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 03. 6月 2025 14:09
+%%%-------------------------------------------------------------------
+-module(efka_monitor).
+-author("anlicheng").
+
+%% API
+-export([]).
+
+%% API
+-export([memory_top/1, cpu_top/1, stop/0]).
+
+memory_top(Interval) when is_integer(Interval) ->
+    spawn(fun()->etop:start([{output, text}, {interval, Interval}, {lines, 20}, {sort, memory}])end).
+
+cpu_top(Interval) when is_integer(Interval) ->
+    spawn(fun()->etop:start([{output, text}, {interval, Interval}, {lines, 20}, {sort, runtime}])end).
+
+stop() ->
+    etop:stop().

apps/efka/src/efka_remote_agent.erl

@@ -0,0 +1,436 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 21. 5月 2025 18:38
+%%%-------------------------------------------------------------------
+-module(efka_remote_agent).
+-author("anlicheng").
+-include("message_pb.hrl").
+-include("efka.hrl").
+-include("efka_tables.hrl").
+
+-behaviour(gen_statem).
+
+%% API
+-export([start_link/0]).
+-export([metric_data/4, event/3, ping/13, request_service_config/2, await_reply/2]).
+
+%% gen_statem callbacks
+-export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]).
+
+-define(SERVER, ?MODULE).
+
+%% 标记当前agent的状态，只有在 activated 状态下才可以正常的发送数据
+-define(STATE_DENIED, denied).
+-define(STATE_CONNECTING, connecting).
+-define(STATE_AUTH, auth).
+%% 不能推送消息到服务，但是可以接受服务器的部分指令
+-define(STATE_RESTRICTED, restricted).
+%% 激活状态下
+-define(STATE_ACTIVATED, activated).
+
+-record(state, {
+    transport_pid :: undefined | pid(),
+    transport_ref :: undefined | reference(),
+    %% 服务器端推送的消息的未确认列表, 映射关系 #{Ref => PacketId}
+    push_inflight = #{},
+    %% 发送的请求的未确认列表, 映射关系 #{Ref => ReceiverPid}
+    request_inflight = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+%% 发送数据
+-spec metric_data(ServiceId :: binary(), DeviceUUID::binary(), RouteKey :: binary(), Metric :: binary()) -> no_return().
+metric_data(ServiceId, DeviceUUID, RouteKey, Metric) when is_binary(ServiceId), is_binary(DeviceUUID), is_binary(RouteKey), is_binary(Metric) ->
+    gen_statem:cast(?SERVER, {metric_data, ServiceId, DeviceUUID, RouteKey, Metric}).
+
+-spec event(ServiceId :: binary(), EventType :: integer(), Params :: binary()) -> no_return().
+event(ServiceId, EventType, Params) when is_binary(ServiceId), is_integer(EventType), is_binary(Params) ->
+    gen_statem:cast(?SERVER, {event, ServiceId, EventType, Params}).
+
+ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) ->
+    gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}).
+
+%% 请求微服务的配置
+-spec request_service_config(ReceiverPid :: pid(), ServiceId :: binary()) -> {ok, Ref :: reference()} | {error, Reason :: term()}.
+request_service_config(ReceiverPid, ServiceId) when is_binary(ServiceId) ->
+    gen_statem:call(?SERVER, {request_service_config, ReceiverPid, ServiceId}).
+
+%% 等待消息的回复
+-spec await_reply(Ref :: reference(), Timeout :: timeout()) -> {ok, Reply :: binary()} | {error, timeout}.
+await_reply(Ref, Timeout) when is_reference(Ref), is_integer(Timeout) ->
+    receive
+        {request_reply, Ref, ReplyBin} ->
+            {ok, ReplyBin}
+    after Timeout ->
+        {error, timeout}
+    end.
+
+%% @doc Creates a gen_statem process which calls Module:init/1 to
+%% initialize. To ensure a synchronized start-up procedure, this
+%% function does not return until Module:init/1 has returned.
+start_link() ->
+    gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_statem callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Whenever a gen_statem is started using gen_statem:start/[3,4] or
+%% gen_statem:start_link/[3,4], this function is called by the new
+%% process to initialize.
+init([]) ->
+    erlang:start_timer(0, self(), create_transport),
+    {ok, ?STATE_DENIED, #state{}}.
+
+%% @private
+%% @doc This function is called by a gen_statem when it needs to find out
+%% the callback mode of the callback module.
+callback_mode() ->
+    handle_event_function.
+
+%% @private
+%% @doc If callback_mode is handle_event_function, then whenever a
+%% gen_statem receives an event from call/2, cast/2, or as a normal
+%% process message, this function is called.
+handle_event({call, From}, {request_service_config, ReceiverPid, ServiceId}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid, request_inflight = RequestInflight}) ->
+    Ref = efka_transport:request(TransportPid, ?METHOD_REQUEST_SERVICE_CONFIG, ServiceId),
+    {keep_state, State#state{request_inflight = maps:put(Ref, ReceiverPid, RequestInflight)}, [{reply, From, {ok, Ref}}]};
+
+handle_event({call, From}, {request_service_config, _ReceiverPid, _ServiceId}, _, State) ->
+    {keep_state, State, [{reply, From, {error, <<"transport is not alive">>}}]};
+
+%% 异步发送数据, 连接存在时候直接发送；否则缓存到mnesia
+handle_event(cast, {metric_data, ServiceId, DeviceUUID, RouteKey, Metric}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    Packet = message_pb:encode_msg(#data{
+        service_id = ServiceId,
+        device_uuid = DeviceUUID,
+        route_key = RouteKey,
+        metric = Metric
+    }),
+    efka_transport:send(TransportPid, ?METHOD_DATA, Packet),
+    {keep_state, State};
+
+handle_event(cast, {metric_data, ServiceId, DeviceUUID, LineProtocolData}, _, State) ->
+    Packet = message_pb:encode_msg(#data{
+        service_id = ServiceId,
+        device_uuid = DeviceUUID,
+        metric = LineProtocolData
+    }),
+    ok = cache_model:insert(?METHOD_DATA, Packet),
+    {keep_state, State};
+
+%% 异步发送事件
+handle_event(cast, {event, ServiceId, EventType, Params}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    EventPacket = message_pb:encode_msg(#event{
+        service_id = ServiceId,
+        event_type = EventType,
+        params = Params
+    }),
+    efka_transport:send(TransportPid, ?METHOD_EVENT, EventPacket),
+    {keep_state, State};
+handle_event(cast, {event, ServiceId, EventType, Params}, ?STATE_ACTIVATED, State) ->
+    EventPacket = message_pb:encode_msg(#event{
+        service_id = ServiceId,
+        event_type = EventType,
+        params = Params
+    }),
+    ok = cache_model:insert(?METHOD_EVENT, EventPacket),
+    {keep_state, State};
+
+handle_event(cast, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}, ?STATE_ACTIVATED,
+    State = #state{transport_pid = TransportPid}) ->
+
+    Ping = message_pb:encode_msg(#ping{
+        adcode = AdCode,
+        boot_time = BootTime,
+        province = Province,
+        city = City,
+        efka_version = EfkaVersion,
+        kernel_arch = KernelArch,
+        ips = Ips,
+        cpu_core = CpuCore,
+        cpu_load = CpuLoad,
+        cpu_temperature = CpuTemperature,
+        disk = Disk,
+        memory = Memory,
+        interfaces = Interfaces
+    }),
+    efka_transport:send(TransportPid, ?METHOD_PING, Ping),
+    {keep_state, State};
+
+%% 异步建立到服务器的连接
+handle_event(info, {timeout, _, create_transport}, ?STATE_DENIED, State) ->
+    {ok, Props} = application:get_env(efka, tls_server),
+    Host = proplists:get_value(host, Props),
+    Port = proplists:get_value(port, Props),
+    {ok, {TransportPid, TransportRef}} = efka_transport:start_monitor(self(), Host, Port),
+    efka_transport:connect(TransportPid),
+
+    {next_state, ?STATE_CONNECTING, State#state{transport_pid = TransportPid, transport_ref = TransportRef}};
+
+handle_event(info, {connect_reply, Reply}, ?STATE_CONNECTING, State = #state{transport_pid = TransportPid}) ->
+    case Reply of
+        ok ->
+            AuthBin = auth_request(),
+            efka_transport:auth_request(TransportPid, AuthBin),
+            {next_state, ?STATE_AUTH, State};
+        {error, Reason} ->
+            lager:debug("[efka_remote_agent] connect failed, error: ~p, pid: ~p", [Reason, TransportPid]),
+            efka_transport:stop(TransportPid),
+            {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
+    end;
+
+handle_event(info, {auth_reply, Reply}, ?STATE_AUTH, State = #state{transport_pid = TransportPid}) ->
+    case Reply of
+        {ok, ReplyBin} ->
+            #auth_reply{code = Code, message = Message} = message_pb:decode_msg(ReplyBin, auth_reply),
+            case Code of
+                0 ->
+                    lager:debug("[efka_remote_agent] auth success, message: ~p", [Message]),
+                    {next_state, ?STATE_ACTIVATED, State, [{next_event, info, flush_cache}]};
+                1 ->
+                    %% 主机在后台的授权未通过；此时agent不能推送数据给云端服务器，但是云端服务器可以推送命令给agent
+                    %% socket的连接状态需要维持
+                    lager:debug("[efka_remote_agent] auth denied, message: ~p", [Message]),
+                    {next_state, ?STATE_RESTRICTED, State};
+                2 ->
+                    % 其他类型的错误，需要间隔时间重试
+                    lager:debug("[efka_remote_agent] auth failed, message: ~p", [Message]),
+                    efka_transport:stop(TransportPid),
+                    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}};
+                _ ->
+                    % 其他类型的错误，需要间隔时间重试
+                    lager:debug("[efka_remote_agent] auth failed, invalid message"),
+                    efka_transport:stop(TransportPid),
+                    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
+            end;
+        {error, Reason} ->
+            lager:debug("[efka_remote_agent] auth_request failed, error: ~p", [Reason]),
+            efka_transport:stop(TransportPid),
+            {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
+    end;
+
+%% 将缓存中的数据推送到服务器端
+handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    case cache_model:fetch_next() of
+        {ok, #cache{id = Id, method = Method, data = Packet}} ->
+            efka_transport:send(TransportPid, Method, Packet),
+            cache_model:delete(Id),
+            {keep_state, State, [{next_event, info, flush_cache}]};
+        error ->
+            {keep_state, State}
+    end;
+handle_event(info, flush_cache, _, State) ->
+    {keep_state, State};
+
+%% 云端服务器推送了消息
+%% 激活消息
+
+%% 微服务部署
+handle_event(info, {server_async_call, PacketId, <<?PUSH_DEPLOY:8, DeployBin/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #deploy{task_id = TaskId, service_id = ServiceId, tar_url = TarUrl} = message_pb:decode_msg(DeployBin, deploy),
+
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    Reply = case efka_inetd:deploy(TaskId, ServiceId, TarUrl) of
+                ok ->
+                    #async_call_reply{code = 1, result = <<"ok">>};
+                {error, Reason} when is_binary(Reason) ->
+                    #async_call_reply{code = 0, message = Reason}
+            end,
+    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+
+    {keep_state, State};
+
+%% 启动微服务
+handle_event(info, {server_async_call, PacketId, <<?PUSH_START_SERVICE:8, ServiceId/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    Reply = case efka_inetd:start_service(ServiceId) of
+                ok ->
+                    #async_call_reply{code = 1, result = <<"ok">>};
+                {error, Reason} when is_binary(Reason) ->
+                    #async_call_reply{code = 0, message = Reason}
+            end,
+    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+
+    {keep_state, State};
+
+%% 停止微服务
+handle_event(info, {server_async_call, PacketId, <<?PUSH_STOP_SERVICE:8, ServiceId/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    Reply = case efka_inetd:stop_service(ServiceId) of
+                ok ->
+                    #async_call_reply{code = 1, result = <<"ok">>};
+                {error, Reason} when is_binary(Reason) ->
+                    #async_call_reply{code = 0, message = Reason}
+            end,
+    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+
+    {keep_state, State};
+
+%% config.json配置信息
+handle_event(info, {server_async_call, PacketId, <<?PUSH_SERVICE_CONFIG:8, ConfigBin/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid, push_inflight = PushInflight}) ->
+    #push_service_config{service_id = ServiceId, config_json = ConfigJson, timeout = Timeout} = message_pb:decode_msg(ConfigBin, push_service_config),
+
+    case efka_service:get_pid(ServiceId) of
+        undefined ->
+            Reply = #async_call_reply{code = 0, message = <<"service not run">>},
+            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+            {keep_state, State};
+        ServicePid when is_pid(ServicePid) ->
+            Ref = make_ref(),
+            %% 将配置文件推送到对应的微服务
+            efka_service:push_config(ServicePid, Ref, ConfigJson),
+            %% 处理超时逻辑
+            erlang:start_timer(Timeout, self(), {request_timeout, Ref}),
+
+            {keep_state, State#state{push_inflight = maps:put(Ref, PacketId, PushInflight)}}
+    end;
+
+%% 收到需要回复的指令
+handle_event(info, {server_async_call, PacketId, <<?PUSH_INVOKE:8, InvokeBin/binary>>}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) ->
+    #invoke{service_id = ServiceId, payload = Payload, timeout = Timeout} = message_pb:decode_msg(InvokeBin, invoke),
+    %% 消息发送到订阅系统
+    case efka_service:get_pid(ServiceId) of
+        undefined ->
+            Reply = #async_call_reply{code = 0, message = <<"micro_service not run">>},
+            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+
+            {keep_state, State};
+        ServicePid when is_pid(ServicePid) ->
+            Ref = make_ref(),
+            efka_service:invoke(ServicePid, Ref, Payload),
+            %% 处理超时逻辑
+            erlang:start_timer(Timeout, self(), {request_timeout, Ref}),
+
+            {keep_state, State#state{push_inflight = maps:put(Ref, PacketId, PushInflight)}}
+    end;
+
+%% 处理task_log
+handle_event(info, {server_async_call, PacketId, <<?PUSH_TASK_LOG:8, TaskLogBin/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #fetch_task_log{task_id = TaskId} = message_pb:decode_msg(TaskLogBin, fetch_task_log),
+    lager:debug("[efka_remote_agent] get task_log request: ~p", [TaskId]),
+    {ok, Logs} = efka_inetd_task_log:get_logs(TaskId),
+    Reply = case length(Logs) > 0 of
+                true ->
+                    Result = iolist_to_binary(jiffy:encode(Logs, [force_utf8])),
+                    #async_call_reply{code = 1, result = Result};
+                false ->
+                    #async_call_reply{code = 1, result = <<"[]">>}
+            end,
+    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+
+    {keep_state, State};
+
+%% 处理命令
+handle_event(info, {server_command, ?COMMAND_AUTH, <<Auth:8>>}, StateName, State = #state{transport_pid = TransportPid}) ->
+    case {Auth, StateName} of
+        {1, ?STATE_ACTIVATED} ->
+            {keep_state, State};
+        {1, ?STATE_DENIED} ->
+            %% 重新激活, 需要重新校验
+            AuthRequestBin = auth_request(),
+            efka_transport:auth_request(TransportPid, AuthRequestBin),
+            {next_state, ?STATE_AUTH, State};
+        {0, _} ->
+            %% 这个时候的主机应该是受限制的状态，不允许发送消息；但是能够接受服务器推送的消息
+            {next_state, ?STATE_RESTRICTED, State}
+    end;
+
+%% 处理Pub/Sub机制
+handle_event(info, {server_pub, Topic, Content}, ?STATE_ACTIVATED, State) ->
+    lager:debug("[efka_remote_agent] get pub topic: ~p, content: ~p", [Topic, Content]),
+    %% 消息发送到订阅系统
+    efka_subscription:publish(Topic, Content),
+    {keep_state, State};
+
+%% 收到来自efka_service的回复
+handle_event(info, {service_reply, Ref, EmsReply}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) ->
+    case maps:take(Ref, PushInflight) of
+        error ->
+            {keep_state, State};
+        {PacketId, NPushInflight} ->
+            Reply = case EmsReply of
+                        {ok, Result} ->
+                            #async_call_reply{code = 1, result = Result};
+                        {error, Reason} ->
+                            #async_call_reply{code = 0, message = Reason}
+                    end,
+            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+
+            {keep_state, State#state{push_inflight = NPushInflight}}
+    end;
+
+%% 收到来自服务器端的回复
+handle_event(info, {server_reply, Ref, ReplyBin}, ?STATE_ACTIVATED, State = #state{request_inflight = RequestInflight}) ->
+    case maps:take(Ref, RequestInflight) of
+        error ->
+            {keep_state, State};
+        {ReceiverPid, NRequestInflight} ->
+            is_process_alive(ReceiverPid) andalso erlang:send(ReceiverPid, {request_reply, Ref, ReplyBin}),
+            {keep_state, State#state{push_inflight = NRequestInflight}}
+    end;
+
+%% todo 请求超时逻辑处理
+handle_event(info, {timeout, _, {request_timeout, Ref}}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) ->
+    case maps:take(Ref, PushInflight) of
+        error ->
+            {keep_state, State};
+        {PacketId, NPushInflight} ->
+            Reply = #async_call_reply{code = 0, message = <<"reqeust timeout">>, result = <<>>},
+            efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+
+            {keep_state, State#state{push_inflight = NPushInflight}}
+    end;
+
+%% transport进程退出
+handle_event(info, {'DOWN', MRef, process, TransportPid, Reason}, _, State = #state{transport_ref = MRef}) ->
+    lager:debug("[efka_remote_agent] transport pid: ~p, exit with reason: ~p", [TransportPid, Reason]),
+    erlang:start_timer(5000, self(), create_transport),
+    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined, transport_ref = undefined}}.
+
+%% @private
+%% @doc This function is called by a gen_statem when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_statem terminates with
+%% Reason. The return value is ignored.
+terminate(_Reason, _StateName, _State = #state{transport_pid = TransportPid}) ->
+    case is_pid(TransportPid) andalso is_process_alive(TransportPid) of
+        true ->
+            efka_transport:stop(TransportPid);
+        false ->
+            ok
+    end,
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+code_change(_OldVsn, StateName, State = #state{}, _Extra) ->
+    {ok, StateName, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec auth_request() -> binary().
+auth_request() ->
+    {ok, AuthInfo} = application:get_env(efka, auth),
+    UUID = proplists:get_value(uuid, AuthInfo),
+    Username = proplists:get_value(username, AuthInfo),
+    Salt = proplists:get_value(salt, AuthInfo),
+    Token = proplists:get_value(token, AuthInfo),
+
+    message_pb:encode_msg(#auth_request{
+        uuid = unicode:characters_to_binary(UUID),
+        username = unicode:characters_to_binary(Username),
+        salt = unicode:characters_to_binary(Salt),
+        token = unicode:characters_to_binary(Token),
+        timestamp = efka_util:timestamp()
+    }).

apps/efka/src/efka_service.erl

@@ -0,0 +1,291 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%% 1. 需要管理服务的整个生命周期，包括: 启动，停止
+%%% 2. 需要监控服务的状态，通过port的方式
+%%% 3. 服务的启动和关闭，需要在更高的层级控制
+%%% @end
+%%% Created : 18. 4月 2025 16:50
+%%%-------------------------------------------------------------------
+-module(efka_service).
+-author("anlicheng").
+-include("efka_tables.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/2]).
+-export([get_name/1, get_pid/1, attach_channel/2]).
+-export([push_config/3, request_config/1, invoke/3]).
+-export([metric_data/4, send_event/3]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-record(state, {
+    service_id :: binary(),
+    %% 通道id信息
+    channel_pid :: pid() | undefined,
+    %% 当前进程的port信息, OSPid = erlang:port_info(Port, os_pid)
+    port :: undefined | port(),
+    %% 系统对应的pid
+    os_pid :: undefined | integer(),
+    %% 配置信息
+    manifest :: undefined | efka_manifest:manifest(),
+    inflight = #{},
+
+    %% 映射关系: #{Ref => Fun}
+    callbacks = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec get_name(ServiceId :: binary()) -> atom().
+get_name(ServiceId) when is_binary(ServiceId) ->
+    list_to_atom("efka_service:" ++ binary_to_list(ServiceId)).
+
+-spec get_pid(ServiceId :: binary()) -> undefined | pid().
+get_pid(ServiceId) when is_binary(ServiceId) ->
+    whereis(get_name(ServiceId)).
+
+-spec push_config(Pid :: pid(), Ref :: reference(), ConfigJson :: binary()) -> no_return().
+push_config(Pid, Ref, ConfigJson) when is_pid(Pid), is_binary(ConfigJson) ->
+    gen_server:cast(Pid, {push_config, Ref, self(), ConfigJson}).
+
+-spec invoke(Pid :: pid(), Ref :: reference(), Payload :: binary()) -> no_return().
+invoke(Pid, Ref, Payload) when is_pid(Pid), is_reference(Ref), is_binary(Payload) ->
+    gen_server:cast(Pid, {invoke, Ref, self(), Payload}).
+
+-spec request_config(Pid :: pid()) -> {ok, Config :: binary()}.
+request_config(Pid) when is_pid(Pid) ->
+    gen_server:call(Pid, request_config).
+
+-spec metric_data(Pid :: pid(), DeviceUUID :: binary(), RouteKey :: binary(), Metric :: binary()) -> no_return().
+metric_data(Pid, DeviceUUID, RouteKey, Metric) when is_pid(Pid), is_binary(DeviceUUID), is_binary(RouteKey),  is_binary(Metric) ->
+    gen_server:cast(Pid, {metric_data, DeviceUUID, RouteKey, Metric}).
+
+-spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> no_return().
+send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) ->
+    gen_server:cast(Pid, {send_event, EventType, Params}).
+
+-spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}.
+attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) ->
+    gen_server:call(Pid, {attach_channel, ChannelPid}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link(Name :: atom(), Service :: binary()) ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) ->
+    gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([ServiceId]) ->
+    %% supervisor进程通过exit(ChildPid, shutdown)调用的时候，确保terminate函数被调用
+    erlang:process_flag(trap_exit, true),
+    case service_model:get_service(ServiceId) of
+        error ->
+            lager:notice("[efka_service] service_id: ~p, not found", [ServiceId]),
+            ignore;
+        {ok, #service{root_dir = RootDir}} ->
+            %% 第一次启动，要求必须成功；只有第一次启动成功，后续的重启逻辑才有意义
+            case efka_manifest:new(RootDir) of
+                {ok, Manifest} ->
+                    case efka_manifest:startup(Manifest) of
+                        {ok, Port} ->
+                            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
+                            lager:debug("[efka_service] service: ~p, port: ~p, boot_service success os_pid: ~p", [ServiceId, Port, OSPid]),
+                            {ok, #state{service_id = ServiceId, manifest = Manifest, port = Port, os_pid = OSPid}};
+                        {error, Reason} ->
+                            lager:debug("[efka_service] service: ~p, boot_service get error: ~p", [ServiceId, Reason]),
+                            {stop, Reason}
+                    end;
+                {error, Reason} ->
+                    lager:notice("[efka_service] service: ~p, read manifest.json get error: ~p", [ServiceId, Reason]),
+                    ignore
+            end
+    end.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 绑定channel
+handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) ->
+    case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of
+        false ->
+            erlang:monitor(process, ChannelPid),
+            lager:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
+            {reply, ok, State#state{channel_pid = ChannelPid}};
+        true ->
+            {reply, {error, <<"channel exists">>}, State}
+    end;
+
+%% 请求参数项 done
+handle_call(request_config, _From, State = #state{service_id = ServiceId}) ->
+    case service_model:get_config_json(ServiceId) of
+        {ok, ConfigJson} ->
+            {reply, {ok, ConfigJson}, State};
+        error ->
+            {reply, {ok, <<>>}, State}
+    end;
+
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast({metric_data, DeviceUUID, RouteKey, Metric}, State = #state{service_id = ServiceId}) ->
+    lager:debug("[efka_service] metric_data service_id: ~p, device_uuid: ~p, route_key: ~p, metric data: ~p", [ServiceId, DeviceUUID, RouteKey, Metric]),
+    efka_remote_agent:metric_data(ServiceId, DeviceUUID, RouteKey, Metric),
+    {noreply, State};
+
+handle_cast({send_event, EventType, Params}, State = #state{service_id = ServiceId}) ->
+    efka_remote_agent:event(ServiceId, EventType, Params),
+    lager:debug("[efka_service] send_event, service_id: ~p, event_type: ~p, params: ~p", [ServiceId, EventType, Params]),
+    {noreply, State};
+
+%% 推送配置项目
+handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{channel_pid = ChannelPid, service_id = ServiceId, inflight = Inflight, callbacks = Callbacks}) ->
+    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
+        true ->
+            gen_channel:push_config(ChannelPid, Ref, self(), ConfigJson),
+            %% 设置成功，需要更新微服务的配置
+            CB = fun() -> service_model:set_config(ServiceId, ConfigJson) end,
+            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight), callbacks = maps:put(Ref, CB, Callbacks)}};
+        false ->
+            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
+            {noreply, State}
+    end;
+
+%% 推送配置项目
+handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{channel_pid = ChannelPid, inflight = Inflight}) ->
+    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
+        true ->
+            gen_channel:invoke(ChannelPid, Ref, self(), Payload),
+            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight)}};
+        false ->
+            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
+            {reply, State}
+    end;
+
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 重启服务
+handle_info({timeout, _, reboot_service}, State = #state{service_id = ServiceId, manifest = Manifest}) ->
+    case efka_manifest:startup(Manifest) of
+        {ok, Port} ->
+            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
+            lager:debug("[efka_service] service_id: ~p, reboot success, port: ~p, os_pid: ~p", [ServiceId, Port, OSPid]),
+            {noreply, State#state{port = Port, os_pid = OSPid}};
+        {error, Reason} ->
+            lager:debug("[efka_service] service_id: ~p, boot_service get error: ~p", [ServiceId, Reason]),
+            try_reboot(),
+            {noreply, State}
+    end;
+
+%% 处理channel的回复
+handle_info({channel_reply, Ref, Reply}, State = #state{inflight = Inflight, callbacks = Callbacks}) ->
+    case maps:take(Ref, Inflight) of
+        error ->
+            {noreply, State};
+        {ReceiverPid, NInflight} ->
+            ReceiverPid ! {service_reply, Ref, Reply},
+
+            {noreply, State#state{inflight = NInflight, callbacks = trigger_callback(Ref, Callbacks)}}
+    end;
+
+handle_info({Port, {data, Data}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
+    lager:debug("[efka_service] service_id: ~p, port data: ~p", [ServiceId, Data]),
+    {noreply, State};
+
+%% 处理port的消息, Port的被动关闭会触发；因此这个时候的Port和State.port的值是相等的
+handle_info({Port, {exit_status, Code}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
+    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with code: ~p", [ServiceId, Port, Code]),
+    {noreply, State#state{port = undefined, os_pid = undefined}};
+
+%% 处理port的退出消息
+handle_info({'EXIT', Port, Reason}, State = #state{service_id = ServiceId}) when is_port(Port) ->
+    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with reason: ~p", [ServiceId, Port, Reason]),
+    try_reboot(),
+    {noreply, State#state{port = undefined, os_pid = undefined}};
+
+%% 处理channel进程的退出
+handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) ->
+    lager:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
+    {noreply, State#state{channel_pid = undefined, inflight = #{}}}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(Reason, _State = #state{service_id = ServiceId, port = Port, os_pid = OSPid}) ->
+    erlang:is_port(Port) andalso erlang:port_close(Port),
+    catch kill_os_pid(OSPid),
+    lager:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+%% 关闭系统进程
+-spec kill_os_pid(port() | undefined) -> no_return().
+kill_os_pid(undefined) ->
+    ok;
+kill_os_pid(OSPid) when is_integer(OSPid) ->
+    Cmd = lists:flatten(io_lib:format("kill -9 ~p", [OSPid])),
+    lager:debug("kill cmd is: ~p", [Cmd]),
+    os:cmd(Cmd).
+
+-spec try_reboot() -> no_return().
+try_reboot() ->
+    erlang:start_timer(5000, self(), reboot_service).
+
+-spec trigger_callback(Ref :: reference(), Callbacks :: map()) -> NewCallbacks :: map().
+trigger_callback(Ref, Callbacks) ->
+    case maps:take(Ref, Callbacks) of
+        error ->
+            Callbacks;
+        {Fun, NCallbacks} ->
+            catch Fun(),
+            NCallbacks
+    end.

apps/efka/src/efka_service_sup.erl

@@ -39,10 +39,16 @@ start_link() ->
 %% this function is called by the new process to find out about
 %% restart strategy, maximum restart frequency and child
 %% specifications.
--spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
 init([]) ->
     SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
-    {ok, {SupFlags, []}}.
+    %% 简化逻辑，只启动需要运行的微服务
+    {ok, Services} = service_model:get_running_services(),
+    ServiceIds = lists:map(fun(#service{service_id = ServiceId}) -> ServiceId end, Services),
+    lager:debug("[efka_service_sup] will start services: ~p", [ServiceIds]),
+
+    Specs = lists:map(fun(ServiceId) -> child_spec(ServiceId) end, Services),
+
+    {ok, {SupFlags, Specs}}.
 
 %%%===================================================================
 %%% Internal functions
@@ -65,7 +71,8 @@ stop_service(ServiceId) when is_binary(ServiceId) ->
     supervisor:terminate_child(?MODULE, ChildId),
     supervisor:delete_child(?MODULE, ChildId).
 
--spec child_spec(binary()) -> supervisor:child_spec().
+child_spec(#service{service_id = ServiceId}) when is_binary(ServiceId) ->
+    child_spec(ServiceId);
 child_spec(ServiceId) when is_binary(ServiceId) ->
     Name = efka_service:get_name(ServiceId),
     #{

apps/efka/src/efka_std_modbus_service.erl

@@ -0,0 +1,291 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%% 1. 需要管理服务的整个生命周期，包括: 启动，停止
+%%% 2. 需要监控服务的状态，通过port的方式
+%%% 3. 服务的启动和关闭，需要在更高的层级控制
+%%% @end
+%%% Created : 18. 4月 2025 16:50
+%%%-------------------------------------------------------------------
+-module(efka_std_modbus_service).
+-author("anlicheng").
+-include("efka_tables.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/2]).
+-export([get_name/1, get_pid/1, attach_channel/2]).
+-export([push_config/3, request_config/1, invoke/3]).
+-export([metric_data/3, send_event/3]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-record(state, {
+    service_id :: binary(),
+    %% 通道id信息
+    channel_pid :: pid() | undefined,
+    %% 当前进程的port信息, OSPid = erlang:port_info(Port, os_pid)
+    port :: undefined | port(),
+    %% 系统对应的pid
+    os_pid :: undefined | integer(),
+    %% 配置信息
+    manifest :: undefined | efka_manifest:manifest(),
+    inflight = #{},
+
+    %% 映射关系: #{Ref => Fun}
+    callbacks = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec get_name(ServiceId :: binary()) -> atom().
+get_name(ServiceId) when is_binary(ServiceId) ->
+    list_to_atom("efka_service:" ++ binary_to_list(ServiceId)).
+
+-spec get_pid(ServiceId :: binary()) -> undefined | pid().
+get_pid(ServiceId) when is_binary(ServiceId) ->
+    whereis(get_name(ServiceId)).
+
+-spec push_config(Pid :: pid(), Ref :: reference(), ConfigJson :: binary()) -> no_return().
+push_config(Pid, Ref, ConfigJson) when is_pid(Pid), is_binary(ConfigJson) ->
+    gen_server:cast(Pid, {push_config, Ref, self(), ConfigJson}).
+
+-spec invoke(Pid :: pid(), Ref :: reference(), Payload :: binary()) -> no_return().
+invoke(Pid, Ref, Payload) when is_pid(Pid), is_reference(Ref), is_binary(Payload) ->
+    gen_server:cast(Pid, {invoke, Ref, self(), Payload}).
+
+-spec request_config(Pid :: pid()) -> {ok, Config :: binary()}.
+request_config(Pid) when is_pid(Pid) ->
+    gen_server:call(Pid, request_config).
+
+-spec metric_data(Pid :: pid(), DeviceUUID :: binary(), Data :: binary()) -> no_return().
+metric_data(Pid, DeviceUUID, Data) when is_pid(Pid), is_binary(DeviceUUID), is_binary(Data) ->
+    gen_server:cast(Pid, {metric_data, DeviceUUID, Data}).
+
+-spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> no_return().
+send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) ->
+    gen_server:cast(Pid, {send_event, EventType, Params}).
+
+-spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}.
+attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) ->
+    gen_server:call(Pid, {attach_channel, ChannelPid}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link(Name :: atom(), Service :: binary()) ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) ->
+    gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([ServiceId]) ->
+    %% supervisor进程通过exit(ChildPid, shutdown)调用的时候，确保terminate函数被调用
+    erlang:process_flag(trap_exit, true),
+    case service_model:get_service(ServiceId) of
+        error ->
+            lager:notice("[efka_service] service_id: ~p, not found", [ServiceId]),
+            ignore;
+        {ok, #service{root_dir = RootDir}} ->
+            %% 第一次启动，要求必须成功；只有第一次启动成功，后续的重启逻辑才有意义
+            case efka_manifest:new(RootDir) of
+                {ok, Manifest} ->
+                    case efka_manifest:startup(Manifest) of
+                        {ok, Port} ->
+                            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
+                            lager:debug("[efka_service] service: ~p, port: ~p, boot_service success os_pid: ~p", [ServiceId, Port, OSPid]),
+                            {ok, #state{service_id = ServiceId, manifest = Manifest, port = Port, os_pid = OSPid}};
+                        {error, Reason} ->
+                            lager:debug("[efka_service] service: ~p, boot_service get error: ~p", [ServiceId, Reason]),
+                            {stop, Reason}
+                    end;
+                {error, Reason} ->
+                    lager:notice("[efka_service] service: ~p, read manifest.json get error: ~p", [ServiceId, Reason]),
+                    ignore
+            end
+    end.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 绑定channel
+handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) ->
+    case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of
+        false ->
+            erlang:monitor(process, ChannelPid),
+            lager:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
+            {reply, ok, State#state{channel_pid = ChannelPid}};
+        true ->
+            {reply, {error, <<"channel exists">>}, State}
+    end;
+
+%% 请求参数项 done
+handle_call(request_config, _From, State = #state{service_id = ServiceId}) ->
+    case service_model:get_config_json(ServiceId) of
+        {ok, ConfigJson} ->
+            {reply, {ok, ConfigJson}, State};
+        error ->
+            {reply, {ok, <<>>}, State}
+    end;
+
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast({metric_data, DeviceUUID, LineProtocolData}, State = #state{service_id = ServiceId}) ->
+    lager:debug("[efka_service] metric_data service_id: ~p, device_uuid: ~p, metric data: ~p", [ServiceId, DeviceUUID, LineProtocolData]),
+    efka_remote_agent:metric_data(ServiceId, DeviceUUID, LineProtocolData),
+    {noreply, State};
+
+handle_cast({send_event, EventType, Params}, State = #state{service_id = ServiceId}) ->
+    efka_remote_agent:event(ServiceId, EventType, Params),
+    lager:debug("[efka_service] send_event, service_id: ~p, event_type: ~p, params: ~p", [ServiceId, EventType, Params]),
+    {noreply, State};
+
+%% 推送配置项目
+handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{channel_pid = ChannelPid, service_id = ServiceId, inflight = Inflight, callbacks = Callbacks}) ->
+    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
+        true ->
+            efka_tcp_channel:push_config(ChannelPid, Ref, self(), ConfigJson),
+            %% 设置成功，需要更新微服务的配置
+            CB = fun() -> service_model:set_config(ServiceId, ConfigJson) end,
+            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight), callbacks = maps:put(Ref, CB, Callbacks)}};
+        false ->
+            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
+            {noreply, State}
+    end;
+
+%% 推送配置项目
+handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{channel_pid = ChannelPid, inflight = Inflight}) ->
+    case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of
+        true ->
+            efka_tcp_channel:invoke(ChannelPid, Ref, self(), Payload),
+            {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight)}};
+        false ->
+            ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}},
+            {reply, State}
+    end;
+
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 重启服务
+handle_info({timeout, _, reboot_service}, State = #state{service_id = ServiceId, manifest = Manifest}) ->
+    case efka_manifest:startup(Manifest) of
+        {ok, Port} ->
+            {os_pid, OSPid} = erlang:port_info(Port, os_pid),
+            lager:debug("[efka_service] service_id: ~p, reboot success, port: ~p, os_pid: ~p", [ServiceId, Port, OSPid]),
+            {noreply, State#state{port = Port, os_pid = OSPid}};
+        {error, Reason} ->
+            lager:debug("[efka_service] service_id: ~p, boot_service get error: ~p", [ServiceId, Reason]),
+            try_reboot(),
+            {noreply, State}
+    end;
+
+%% 处理channel的回复
+handle_info({channel_reply, Ref, Reply}, State = #state{inflight = Inflight, callbacks = Callbacks}) ->
+    case maps:take(Ref, Inflight) of
+        error ->
+            {noreply, State};
+        {ReceiverPid, NInflight} ->
+            ReceiverPid ! {service_reply, Ref, Reply},
+
+            {noreply, State#state{inflight = NInflight, callbacks = trigger_callback(Ref, Callbacks)}}
+    end;
+
+handle_info({Port, {data, Data}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
+    lager:debug("[efka_service] service_id: ~p, port data: ~p", [ServiceId, Data]),
+    {noreply, State};
+
+%% 处理port的消息, Port的被动关闭会触发；因此这个时候的Port和State.port的值是相等的
+handle_info({Port, {exit_status, Code}}, State = #state{service_id = ServiceId}) when is_port(Port) ->
+    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with code: ~p", [ServiceId, Port, Code]),
+    {noreply, State#state{port = undefined, os_pid = undefined}};
+
+%% 处理port的退出消息
+handle_info({'EXIT', Port, Reason}, State = #state{service_id = ServiceId}) when is_port(Port) ->
+    lager:debug("[efka_service] service_id: ~p, port: ~p, exit with reason: ~p", [ServiceId, Port, Reason]),
+    try_reboot(),
+    {noreply, State#state{port = undefined, os_pid = undefined}};
+
+%% 处理channel进程的退出
+handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) ->
+    lager:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
+    {noreply, State#state{channel_pid = undefined, inflight = #{}}}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(Reason, _State = #state{service_id = ServiceId, port = Port, os_pid = OSPid}) ->
+    erlang:is_port(Port) andalso erlang:port_close(Port),
+    catch kill_os_pid(OSPid),
+    lager:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+%% 关闭系统进程
+-spec kill_os_pid(port() | undefined) -> no_return().
+kill_os_pid(undefined) ->
+    ok;
+kill_os_pid(OSPid) when is_integer(OSPid) ->
+    Cmd = lists:flatten(io_lib:format("kill -9 ~p", [OSPid])),
+    lager:debug("kill cmd is: ~p", [Cmd]),
+    os:cmd(Cmd).
+
+-spec try_reboot() -> no_return().
+try_reboot() ->
+    erlang:start_timer(5000, self(), reboot_service).
+
+-spec trigger_callback(Ref :: reference(), Callbacks :: map()) -> NewCallbacks :: map().
+trigger_callback(Ref, Callbacks) ->
+    case maps:take(Ref, Callbacks) of
+        error ->
+            Callbacks;
+        {Fun, NCallbacks} ->
+            catch Fun(),
+            NCallbacks
+    end.

apps/efka/src/efka_subscription.erl

@@ -13,7 +13,7 @@
 
 %% API
 -export([start_link/0]).
--export([subscribe/2, unsubscribe_all/1, publish/3, debug_info/0]).
+-export([subscribe/2, publish/2]).
 -export([match_components/2, is_valid_components/1, of_components/1]).
 
 %% gen_server callbacks
@@ -34,30 +34,20 @@
 }).
 
 -record(state, {
-    subscribers = [],
+    subscribers = []
-    %% qos未1，并且未被消费的消息
-    remaining_messages = []
 }).
 
 %%%===================================================================
 %%% API
 %%%===================================================================
 
--spec subscribe(Topic :: binary(), SubscriberPid :: pid()) -> ok | {error, Reason :: binary()}.
+-spec subscribe(Topic :: binary(), SubscriberPid :: pid()) -> no_return().
 subscribe(Topic, SubscriberPid) when is_binary(Topic), is_pid(SubscriberPid) ->
-    gen_server:call(?SERVER, {subscribe, Topic, SubscriberPid}).
+    gen_server:cast(?SERVER, {subscribe, Topic, SubscriberPid}).
 
--spec publish(Topic :: binary(), Qos :: integer(), Content :: binary()) -> ok.
+-spec publish(Topic :: binary(), Content :: binary()) -> no_return().
-publish(Topic, Qos, Content) when is_binary(Topic), is_integer(Qos), is_binary(Content) ->
+publish(Topic, Content) when is_binary(Topic), is_binary(Content) ->
-    gen_server:cast(?SERVER, {publish, Topic, Qos, Content}).
+    gen_server:cast(?SERVER, {publish, Topic, Content}).
-
--spec unsubscribe_all(pid()) -> ok.
-unsubscribe_all(SubscriberPid) when is_pid(SubscriberPid) ->
-    gen_server:call(?SERVER, {unsubscribe_all, SubscriberPid}).
-
--spec debug_info() -> {ok, Info :: map()}.
-debug_info() ->
-    gen_server:call(?SERVER, debug_info).
 
 %% @doc Spawns the server and registers the local name (unique)
 -spec(start_link() ->
@@ -87,38 +77,8 @@ init([]) ->
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
     {stop, Reason :: term(), NewState :: #state{}}).
-%%  同一个SubscriberPid只能订阅同一个topic一次
+handle_call(_Request, _From, State = #state{}) ->
-handle_call({subscribe, Topic, SubscriberPid}, _From, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
+    {reply, ok, State}.
-    Components = of_components(Topic),
-    case is_valid_components(Components) of
-        true ->
-            case has_subscription(Topic, SubscriberPid, Subscribers) of
-                true ->
-                    {reply, ok, State};
-                false ->
-                    Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)},
-                    %% 只有首次看到该pid时才建立monitor，避免重复monitor
-                    case has_subscriber_pid(SubscriberPid, Subscribers) of
-                        true ->
-                            ok;
-                        false ->
-                            erlang:monitor(process, SubscriberPid)
-                    end,
-                    %% 处理遗留的消息
-                    RestRemainingMessages = dispatch_remaining_messages(Sub, RemainingMessages),
-                    {reply, ok, State#state{subscribers = Subscribers ++ [Sub], remaining_messages = RestRemainingMessages}}
-            end;
-        false ->
-            {reply, {error, <<"invalid topic name">>}, State}
-    end;
-handle_call({unsubscribe_all, SubscriberPid}, _From, State = #state{subscribers = Subscribers}) ->
-    {reply, ok, State#state{subscribers = remove_subscriber_pid(SubscriberPid, Subscribers)}};
-handle_call(debug_info, _From, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
-    Info = #{
-        subscribes => Subscribers,
-        remaining_messages => RemainingMessages
-    },
-    {reply, {ok, Info}, State}.
 
 %% @private
 %% @doc Handling cast messages
@@ -126,19 +86,29 @@ handle_call(debug_info, _From, State = #state{subscribers = Subscribers, remaini
     {noreply, NewState :: #state{}} |
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
+%%  同一个SubscriberPid只能订阅同一个topic一次
+handle_cast({subscribe, Topic, SubscriberPid}, State = #state{subscribers = Subscribers}) ->
+    Components = of_components(Topic),
+    case is_valid_components(Components) of
+        true ->
+            Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)},
+            %% 建立到SubscriberPid的monitor，进程退出需要清理订阅
+            erlang:monitor(process, SubscriberPid),
+
+            {noreply, State#state{subscribers = Subscribers ++ [Sub]}};
+        false ->
+            {noreply, State}
+    end;
+
 %% 发布消息
-handle_cast({publish, Topic, Qos, Content}, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
+handle_cast({publish, Topic, Content}, State = #state{subscribers = Subscribers}) ->
     MatchedSubscribers = match_subscribers(Subscribers, Topic),
-    logger:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]),
+    lists:foreach(fun(#subscriber{subscriber_pid = SubscriberPid}) ->
-    case MatchedSubscribers of
+        SubscriberPid ! {topic_broadcast, Topic, Content}
-        [_|_] ->
+                  end, MatchedSubscribers),
-            broadcast(Topic, Content, MatchedSubscribers),
+
-            {noreply, State};
+    lager:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]),
-        [] when Qos =:= 0 ->
+    {noreply, State}.
-            {noreply, State};
-        [] ->
-            {noreply, State#state{remaining_messages = [{Topic, Content}|RemainingMessages]}}
-    end.
 
 %% @private
 %% @doc Handling all non call/cast messages
@@ -147,12 +117,12 @@ handle_cast({publish, Topic, Qos, Content}, State = #state{subscribers = Subscri
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
 handle_info({'DOWN', _Ref, process, SubscriberPid, Reason}, State = #state{subscribers = Subscribers}) ->
-    logger:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]),
+    lager:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]),
-    NSubscribers = remove_subscriber_pid(SubscriberPid, Subscribers),
+    NSubscribers = lists:filter(fun(#subscriber{subscriber_pid = Pid0}) -> SubscriberPid /= Pid0 end, Subscribers),
     {noreply, State#state{subscribers = NSubscribers}};
 
 handle_info(Info, State = #state{}) ->
-    logger:debug("[efka_subscription] get unknown info: ~p", [Info]),
+    lager:debug("[efka_subscription] get unknown info: ~p", [Info]),
     {noreply, State}.
 
 %% @private
@@ -194,17 +164,6 @@ match_subscribers(Subscribers, Topic) when is_list(Subscribers), is_binary(Topic
 contain_channel(Pid, Subscribers) when is_pid(Pid), is_list(Subscribers) ->
     lists:search(fun(#subscriber{subscriber_pid = Pid0}) -> Pid == Pid0 end, Subscribers) /= false.
 
--spec has_subscriber_pid(pid(), [#subscriber{}]) -> boolean().
-has_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) ->
-    lists:any(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =:= SubscriberPid0 end, Subscribers).
-
--spec has_subscription(binary(), pid(), [#subscriber{}]) -> boolean().
-has_subscription(Topic, SubscriberPid, Subscribers)
-  when is_binary(Topic), is_pid(SubscriberPid), is_list(Subscribers) ->
-    lists:any(fun(#subscriber{topic = Topic0, subscriber_pid = SubscriberPid0}) ->
-        Topic =:= Topic0 andalso SubscriberPid =:= SubscriberPid0
-    end, Subscribers).
-
 %% 开始对比订阅的topic和发布的topic的Components信息
 %% *表示单级匹配，+表示多级匹配；+只能出现一次，并且只能在末尾
 -spec match_components(list(), list()) -> boolean().
@@ -225,7 +184,6 @@ match_components(_, _, _) ->
 of_components(Topic) when is_binary(Topic) ->
     binary:split(Topic, <<$/>>, [global]).
 
--spec is_valid_components([binary()]) -> boolean().
 is_valid_components([]) ->
     true;
 is_valid_components([<<$+>>|T]) ->
@@ -244,27 +202,3 @@ order_num([<<$+>>|_]) ->
     3;
 order_num([_|Tail]) ->
     order_num(Tail).
-
--spec broadcast(binary(), binary(), [#subscriber{}]) -> ok.
-broadcast(Topic, Content, MatchedSubscribers) ->
-    lists:foreach(fun(#subscriber{subscriber_pid = SubscriberPid}) ->
-        SubscriberPid ! {topic_broadcast, Topic, Content}
-                  end, MatchedSubscribers).
-
--spec remove_subscriber_pid(pid(), [#subscriber{}]) -> [#subscriber{}].
-remove_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) ->
-    lists:filter(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =/= SubscriberPid0 end, Subscribers).
-
--spec dispatch_remaining_messages(Subscriber :: #subscriber{}, RemainingMessages :: list()) -> RestRemainingMessages :: list().
-dispatch_remaining_messages(#subscriber{subscriber_pid = SubscriberPid, components = Components}, RemainingMessages) when is_list(RemainingMessages) ->
-    %% 处理遗留的消息
-    lists:foldl(fun({Topic0, Content0}, Acc) ->
-        Components0 = of_components(Topic0),
-        case match_components(Components, Components0) of
-            true ->
-                SubscriberPid ! {topic_broadcast, Topic0, Content0},
-                Acc;
-            false ->
-                [{Topic0, Content0}|Acc]
-        end
-                end, [], RemainingMessages).

apps/efka/src/efka_sup.erl

@@ -13,7 +13,6 @@
 
 -define(SERVER, ?MODULE).
 
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
 start_link() ->
     supervisor:start_link({local, ?SERVER}, ?MODULE, []).
 
@@ -26,35 +25,25 @@ start_link() ->
 %%                  shutdown => shutdown(), % optional
 %%                  type => worker(),       % optional
 %%                  modules => modules()}   % optional
--spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
 init([]) ->
     SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
     ChildSpecs = [
         #{
-            id => 'efka_logger',
+            id => 'efka_model_sup',
-            start => {'efka_logger', start_link, ["deploy_log"]},
+            start => {'efka_model_sup', start_link, []},
-            restart => permanent,
-            shutdown => 2000,
-            type => worker,
-            modules => ['efka_logger']
-        },
-
-        #{
-            id => 'efka_service_sup',
-            start => {'efka_service_sup', start_link, []},
             restart => permanent,
             shutdown => 2000,
             type => supervisor,
-            modules => ['efka_service_sup']
+            modules => ['efka_model_sup']
         },
 
         #{
-            id => efka_service_model,
+            id => 'efka_inetd_task_log',
-            start => {efka_service_model, start_link, []},
+            start => {'efka_inetd_task_log', start_link, []},
             restart => permanent,
-            shutdown => 5000,
+            shutdown => 2000,
             type => worker,
-            modules => ['efka_service_model']
+            modules => ['efka_inetd_task_log']
         },
 
         #{
@@ -67,25 +56,43 @@ init([]) ->
         },
 
         #{
-            id => 'efka_iot_client',
+            id => 'efka_inetd',
-            start => {'efka_iot_client', start_link, []},
+            start => {'efka_inetd', start_link, []},
             restart => permanent,
             shutdown => 2000,
             type => worker,
-            modules => ['efka_iot_client']
+            modules => ['efka_inetd']
         },
 
         #{
-            id => 'efka_iot_heartbeat',
+            id => 'efka_remote_agent',
-            start => {'efka_iot_heartbeat', start_link, []},
+            start => {'efka_remote_agent', start_link, []},
             restart => permanent,
             shutdown => 2000,
             type => worker,
-            modules => ['efka_iot_heartbeat']
+            modules => ['efka_remote_agent']
-        }
+        },
 
+        #{
+            id => 'tcp_sup',
+            start => {'tcp_sup', start_link, []},
+            restart => permanent,
+            shutdown => 2000,
+            type => supervisor,
+            modules => ['tcp_sup']
+        },
+
+        #{
+            id => 'efka_service_sup',
+            start => {'efka_service_sup', start_link, []},
+            restart => permanent,
+            shutdown => 2000,
+            type => supervisor,
+            modules => ['efka_service_sup']
+        }
     ],
 
     {ok, {SupFlags, ChildSpecs}}.
 
 %% internal functions
+

apps/efka/src/efka_transport.erl

@@ -0,0 +1,226 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 20. 4月 2025 18:47
+%%%-------------------------------------------------------------------
+-module(efka_transport).
+-author("anlicheng").
+-include("message_pb.hrl").
+-include("efka.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_monitor/3]).
+-export([connect/1, auth_request/2, send/3, async_call_reply/3, stop/1]).
+-export([request/3]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE). 
+
+-record(state, {
+    parent_pid :: pid(),
+    host :: string(),
+    port :: integer(),
+    socket :: undefined | ssl:sslsocket(),
+    packet_id = 1,
+    %% 通过packet_id建立请求和响应的关系
+    inflight = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec auth_request(Pid :: pid(), AuthBin :: binary()) -> no_return().
+auth_request(Pid, AuthBin) when is_pid(Pid), is_binary(AuthBin) ->
+    gen_server:cast(Pid, {auth_request, AuthBin}).
+
+-spec request(Pid :: pid(), Method :: integer(), ReqBin :: binary()) -> Ref :: reference().
+request(Pid, Method, ReqBin) when is_pid(Pid), is_binary(ReqBin) ->
+    Ref = make_ref(),
+    gen_server:cast(Pid, {request, Ref, Method, ReqBin}),
+    Ref.
+
+-spec connect(Pid :: pid()) -> no_return().
+connect(Pid) when is_pid(Pid) ->
+    gen_server:cast(Pid, connect).
+
+-spec send(Pid :: pid(), Method :: integer(), Packet :: binary()) -> no_return().
+send(Pid, Method, Packet) when is_pid(Pid), is_integer(Method), is_binary(Packet) ->
+    gen_server:cast(Pid, {send, Method, Packet}).
+
+-spec async_call_reply(Pid :: pid() | undefined, PacketId :: integer(), Response :: binary()) -> no_return().
+async_call_reply(undefined, PacketId, Response) when is_integer(PacketId), is_binary(Response) ->
+    ok;
+async_call_reply(Pid, PacketId, Response) when is_pid(Pid), is_integer(PacketId), is_binary(Response) ->
+    gen_server:cast(Pid, {async_call_reply, PacketId, Response}).
+
+%% 关闭的时候不一定能成功，可能关闭的时候；transport进程已经退出了
+-spec stop(Pid :: pid() | undefined) -> ok.
+stop(undefined) ->
+    ok;
+stop(Pid) when is_pid(Pid) ->
+    catch gen_server:stop(Pid, normal, 2000).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_monitor(ParentPid :: pid(), Host :: string(), Port :: integer()) ->
+    {ok, {Pid :: pid(), MRef :: reference()}} | ignore | {error, Reason :: term()}).
+start_monitor(ParentPid, Host, Port) when is_pid(ParentPid), is_list(Host), is_integer(Port) ->
+    gen_server:start_monitor(?MODULE, [ParentPid, Host, Port], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([ParentPid, Host, Port]) ->
+    {ok, #state{parent_pid = ParentPid, host = Host, port = Port, socket = undefined}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call(_Req, _From, State = #state{}) ->
+    {reply, ok, State#state{}}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 建立到目标服务器的连接
+handle_cast(connect, State = #state{host = Host, port = Port, parent_pid = ParentPid}) ->
+    SslOptions = [
+        binary,
+        {packet, 4},
+        {verify, verify_none}
+    ],
+    case ssl:connect(Host, Port, SslOptions, 5000) of
+        {ok, Socket} ->
+            ok = ssl:controlling_process(Socket, self()),
+            ParentPid ! {connect_reply, ok},
+            ping_ticker(),
+            {noreply, State#state{socket = Socket}};
+        {error, Reason} ->
+            ParentPid ! {connect_reply, {error, Reason}},
+            {noreply, State#state{socket = undefined}}
+    end;
+
+%% auth校验
+handle_cast({auth_request, AuthRequestBin}, State = #state{parent_pid = ParentPid, socket = Socket, packet_id = PacketId}) ->
+    ok = ssl:send(Socket, <<?PACKET_REQUEST, PacketId:32, ?METHOD_AUTH, AuthRequestBin/binary>>),
+    %% 需要等待auth返回的结果
+    receive
+        {ssl, Socket, <<?PACKET_RESPONSE, PacketId:32, ReplyBin/binary>>} ->
+            ParentPid ! {auth_reply, {ok, ReplyBin}},
+            {noreply, State#state{packet_id = PacketId + 1}};
+        {ssl, Socket, Info} ->
+            lager:warning("[efka_transport] get invalid auth_reply: ~p", [Info]),
+            ParentPid ! {auth_reply, {error, invalid_auth_reply}},
+            {noreply, State#state{packet_id = PacketId + 1}}
+    after 5000 ->
+        ParentPid ! {auth_reply, {error, timeout}},
+        {noreply, State#state{packet_id = PacketId + 1}}
+    end;
+
+%% 提交请求
+handle_cast({request, Ref, Method, ReqBin}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) ->
+    ok = ssl:send(Socket, <<?PACKET_REQUEST, PacketId:32, Method:8, ReqBin/binary>>),
+    {noreply, State#state{packet_id = PacketId + 1, inflight = maps:put(PacketId, Ref, Inflight)}};
+
+handle_cast({send, Method, Packet}, State = #state{socket = Socket}) ->
+    ok = ssl:send(Socket, <<?PACKET_REQUEST, Method:8, Packet/binary>>),
+    {noreply, State};
+
+%% 服务push的消息的回复
+handle_cast({async_call_reply, PacketId, Response}, State = #state{socket = Socket}) ->
+    ok = ssl:send(Socket, <<?PACKET_ASYNC_CALL_REPLY, PacketId:32, Response/binary>>),
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 服务器主动推送的数据，有packetId的是要求返回的；为0的表示不需要返回值
+handle_info({ssl, Socket, <<?PACKET_COMMAND, CommandType:8, Command/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
+    ParentPid ! {server_command, CommandType, Command},
+    {noreply, State};
+
+handle_info({ssl, Socket, <<?PACKET_PUB, PubBin/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
+    #pub{topic = Topic, content = Content} = message_pb:decode_msg(PubBin, pub),
+    ParentPid ! {server_pub, Topic, Content},
+    {noreply, State};
+
+handle_info({ssl, Socket, <<?PACKET_ASYNC_CALL, PacketId:32, AsyncCallBin/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
+    ParentPid ! {server_async_call, PacketId, AsyncCallBin},
+    {noreply, State};
+
+%% efka:request <-> iot:response
+handle_info({ssl, Socket, <<?PACKET_RESPONSE, PacketId:32, ReplyBin/binary>>}, State = #state{socket = Socket, inflight = Inflight, parent_pid = ParentPid}) ->
+    case maps:take(PacketId, Inflight) of
+        error ->
+            {noreply, State};
+        {Ref, NInflight} ->
+            ParentPid ! {server_reply, Ref, ReplyBin},
+            {noreply, State#state{inflight = NInflight}}
+    end;
+
+handle_info({ssl_error, Socket, Reason}, State = #state{socket = Socket}) ->
+    lager:debug("[efka_transport] ssl error: ~p", [Reason]),
+    {stop, normal, State};
+
+handle_info({ssl_closed, Socket}, State = #state{socket = Socket}) ->
+    {stop, normal, State};
+
+handle_info({timeout, _, ping_ticker}, State = #state{socket = Socket}) ->
+    ok = ssl:send(Socket, <<?PACKET_PING>>),
+    ping_ticker(),
+    {noreply, State};
+
+handle_info(Info, State = #state{}) ->
+    lager:notice("[efka_transport] get unknown info: ~p", [Info]),
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+        State :: #state{}) -> term()).
+terminate(Reason, #state{}) ->
+    lager:notice("[efka_transport] terminate with reason: ~p", [Reason]),
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+        Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+ping_ticker() ->
+    erlang:start_timer(5000, self(), ping_ticker).

apps/efka/src/efka_util.erl

@@ -14,32 +14,26 @@
 -export([timestamp/0, number_format/2, timestamp_ms/0, float_to_binary/2, int_format/2]).
 -export([chunks/2, rand_bytes/1, uuid/0, md5/1, sha_uuid/0]).
 -export([json_data/1, json_error/2]).
--export([starts_with/2, file_md5/1]).
 
--spec get_file_md5(string()) -> string().
 get_file_md5(FilePath) when is_list(FilePath) ->
     {ok, FileData} = file:read_file(FilePath),
     Md5Binary = crypto:hash(md5, FileData),
     string:lowercase(lists:flatten([io_lib:format("~2.16.0b", [X]) || X <- binary_to_list(Md5Binary)])).
 
 %% 时间，精确到毫秒
--spec timestamp_ms() -> integer().
 timestamp_ms() ->
     {Mega, Seconds, Micro} = os:timestamp(),
     (Mega * 1000000 + Seconds) * 1000 + Micro div 1000.
 
--spec timestamp() -> integer().
 timestamp() ->
     {Mega, Seconds, _Micro} = os:timestamp(),
     Mega * 1000000 + Seconds.
 
--spec number_format(integer() | float(), integer()) -> integer() | float().
 number_format(Num, _Decimals) when is_integer(Num) ->
     Num;
 number_format(Float, Decimals) when is_float(Float) ->
     list_to_float(float_to_list(Float, [{decimals, Decimals}, compact])).
 
--spec int_format(integer(), pos_integer()) -> integer().
 int_format(Num, Len) when is_integer(Num), Len > 0 ->
     S = integer_to_list(Num),
     case length(S) > Len of
@@ -55,8 +49,6 @@ chunks(List, Size) when is_list(List), is_integer(Size), Size > 0, length(List)
     [List];
 chunks(List, Size) when is_list(List), is_integer(Size), Size > 0 ->
     chunks0(List, Size, Size, [], []).
-
--spec chunks0(list(), integer(), integer(), list(), [list()]) -> [list()].
 chunks0([], _, _, [], AccTarget) ->
     lists:reverse(AccTarget);
 chunks0([], _, _, Target, AccTarget) ->
@@ -66,22 +58,19 @@ chunks0(List, Size, 0, Target, AccTarget) ->
 chunks0([Hd | Tail], Size, Num, Target, AccTarget) ->
     chunks0(Tail, Size, Num - 1, [Hd | Target], AccTarget).
 
--spec json_data(term()) -> binary().
 json_data(Data) ->
-    iolist_to_binary(json:encode(#{
+    jiffy:encode(#{
         <<"result">> => Data
-    })).
+    }, [force_utf8]).
 
--spec json_error(integer(), binary()) -> binary().
 json_error(ErrCode, ErrMessage) when is_integer(ErrCode), is_binary(ErrMessage) ->
-    iolist_to_binary(json:encode(#{
+    jiffy:encode(#{
         <<"error">> => #{
             <<"code">> => ErrCode,
             <<"message">> => ErrMessage
         }
-    })).
+    }, [force_utf8]).
 
--spec uuid() -> string().
 uuid() ->
     rand_bytes(16).
 
@@ -96,7 +85,6 @@ rand_bytes(Size) when is_integer(Size), Size > 0 ->
 md5(Str) when is_binary(Str) ->
     list_to_binary(lists:flatten([hex(X) || <<X:4>> <= erlang:md5(Str)])).
 
--spec hex(0..15) -> byte().
 hex(N) when N < 10 ->
     $0 + N;
 hex(N) ->
@@ -115,28 +103,3 @@ sha_uuid() ->
     Salt = crypto:strong_rand_bytes(32),
     Str = string:lowercase(binary:encode_hex(crypto:hash(sha256, Salt))),
     binary:part(Str, 1, 32).
-
--spec starts_with(Binary :: binary(), Prefix :: binary()) -> boolean().
-starts_with(Binary, Prefix) when is_binary(Binary), is_binary(Prefix) ->
-    PrefixSize = byte_size(Prefix),
-    case Binary of
-        <<Prefix:PrefixSize/binary, _Rest/binary>> -> true;
-        _ -> false
-    end.
-
--spec file_md5(FilePath :: string()) -> Md5 :: string().
-file_md5(FilePath) when is_list(FilePath) ->
-    {ok, F} = file:open(FilePath, [read, binary]),
-    Digest = md5_loop(F, crypto:hash_init(md5)),
-    file:close(F),
-    lists:flatten(io_lib:format("~32.16.0b", [binary:decode_unsigned(Digest)])).
-
--spec md5_loop(file:io_device(), crypto:hash_state()) -> binary().
-md5_loop(F, Context) ->
-    %% 每次读取 1MB，可调整块大小
-    case file:read(F, 1024 * 1024) of
-        eof ->
-            crypto:hash_final(Context);
-        {ok, Bin} ->
-            md5_loop(F, crypto:hash_update(Context, Bin))
-    end.

apps/efka/src/gen_channel.erl

@@ -0,0 +1,90 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 27. 8月 2025 15:22
+%%%-------------------------------------------------------------------
+-module(gen_channel).
+-author("anlicheng").
+-include("efka_service.hrl").
+
+-export([register/2]).
+-export([push_config/4, invoke/4, channel_reply/3]).
+-export([next_packet_id/1]).
+-export([json_result/2, json_error/3]).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec push_config(ChannelPid :: pid(), Ref :: reference(), ReceiverPid :: pid(), ConfigJson :: binary()) -> no_return().
+push_config(ChannelPid, Ref, ReceiverPid, ConfigJson) when is_pid(ChannelPid), is_pid(ReceiverPid), is_binary(ConfigJson), is_reference(Ref) ->
+    ChannelPid ! {push_config, Ref, ReceiverPid, ConfigJson}.
+
+-spec invoke(ChannelPid :: pid(), Ref :: reference(), ReceiverPid :: pid(), Payload :: binary()) -> no_return().
+invoke(ChannelPid, Ref, ReceiverPid, Payload) when is_pid(ChannelPid), is_pid(ReceiverPid), is_binary(Payload), is_reference(Ref) ->
+    ChannelPid ! {invoke, Ref, ReceiverPid, Payload}.
+
+%% 超时逻辑处理
+channel_reply(Id, Reply, Inflight) ->
+    case maps:take(Id, Inflight) of
+        error ->
+            Inflight;
+        {{ReceiverPid, Ref}, NInflight} ->
+            case is_pid(ReceiverPid) andalso is_process_alive(ReceiverPid) of
+                true ->
+                    ReceiverPid ! {channel_reply, Ref, Reply};
+                false ->
+                    ok
+            end,
+            NInflight
+    end.
+
+%% 注册
+-spec register(Id :: integer(), ServiceId :: binary()) -> {error, Reply :: binary()} | {ok, Reply :: binary(), ServicePid :: pid()}.
+register(Id, ServiceId) when is_integer(Id), is_binary(ServiceId) ->
+    case efka_service:get_pid(ServiceId) of
+        undefined ->
+            lager:warning("[gen_channel] service_id: ~p, not running", [ServiceId]),
+            Reply = json_error(Id, -1, <<"service not running">>),
+            {error, Reply};
+        ServicePid when is_pid(ServicePid) ->
+            case efka_service:attach_channel(ServicePid, self()) of
+                ok ->
+                    Reply = json_result(Id, <<"ok">>),
+                    erlang:monitor(process, ServicePid),
+                    {ok, Reply, ServicePid};
+                {error, Error} ->
+                    lager:warning("[gen_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]),
+                    Reply = json_error(Id, -1, Error),
+                    {error, Reply}
+            end
+    end.
+
+%% 采用32位编码
+-spec next_packet_id(PacketId :: integer()) -> NextPacketId :: integer().
+next_packet_id(PacketId) when PacketId >= 4294967295 ->
+    1;
+next_packet_id(PacketId) ->
+    PacketId + 1.
+
+-spec json_result(Id :: integer(), Result :: term()) -> binary().
+json_result(Id, Result) when is_integer(Id) ->
+    Response = #{
+        <<"id">> => Id,
+        <<"result">> => Result
+    },
+    jiffy:encode(Response, [force_utf8]).
+
+-spec json_error(Id :: integer(), Code :: integer(), Message :: binary()) -> binary().
+json_error(Id, Code, Message) when is_integer(Id), is_integer(Code), is_binary(Message) ->
+    Response = #{
+        <<"id">> => Id,
+        <<"error">> => #{
+            <<"code">> => Code,
+            <<"message">> => Message
+        }
+    },
+    jiffy:encode(Response, [force_utf8]).

apps/efka/src/iot/efka_iot_client.erl

@@ -1,675 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 20. 4月 2026 00:00
-%%%-------------------------------------------------------------------
--module(efka_iot_client).
--author("anlicheng").
--include("efka_tables.hrl").
--include("message.hrl").
--include("message_pb.hrl").
-
--behaviour(gen_statem).
-
-%% API
--export([start_link/0]).
--export([metric_data/2, ping/13]).
--export([send_stream/2, stream_done/1]).
--export([is_activated/0, dropped_message_count/0]).
-
-%% gen_statem callbacks
--export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]).
-
--define(SERVER, ?MODULE).
-
-%% 标记当前agent的状态，只有在 activated 状态下才可以正常的发送数据
--define(STATE_DISCONNECTED, disconnected).
-%% 等待校验中
--define(STATE_AUTH, auth).
-%% 激活状态下
--define(STATE_ACTIVATED, activated).
--define(SSL_PING_INTERVAL, 30000).
--define(OUTBOX_SEGMENT_RECORD_LIMIT, 2000).
--define(OUTBOX_MAX_SEGMENTS, 5).
--define(OUTBOX_MAX_RECORD_BYTES, 16 * 1024 * 1024).
-
--record(state, {
-    socket :: undefined | ssl:sslsocket(),
-    outbox :: efka_iot_outbox:outbox(),
-    streams = #{},
-    %% 保存当前 auth 请求的 packet id，用来建立 auth 请求和响应的对应关系
-    auth_pkt_id = undefined :: undefined | pos_integer(),
-    next_pkt_id = 1 :: pos_integer(),
-    ping_timer_ref = undefined :: undefined | reference(),
-    dropped_message_count = 0 :: non_neg_integer()
-}).
-
--record(stream_state, {
-    worker_pid :: pid(),
-    monitor_ref :: reference()
-}).
-
--type stream_id() :: pos_integer().
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
-%% 发送数据
--spec metric_data(RouteKey :: binary(), Metric :: binary()) -> ok.
-metric_data(RouteKey, Metric) when is_binary(RouteKey), is_binary(Metric) ->
-    gen_statem:cast(?SERVER, {metric_data, RouteKey, Metric}).
-
--spec send_stream(StreamId :: stream_id(), Body :: term()) -> ok.
-send_stream(StreamId, Body) when is_integer(StreamId), StreamId > 0 ->
-    gen_statem:cast(?SERVER, {send_stream, StreamId, Body}).
-
--spec stream_done(StreamId :: stream_id()) -> ok.
-stream_done(StreamId) when is_integer(StreamId), StreamId > 0 ->
-    gen_statem:cast(?SERVER, {stream_done, StreamId}).
-
--spec is_activated() -> boolean().
-is_activated() ->
-    gen_statem:call(?SERVER, is_activated).
-
--spec dropped_message_count() -> non_neg_integer().
-dropped_message_count() ->
-    gen_statem:call(?SERVER, dropped_message_count).
-
--spec ping(term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term()) -> ok.
-ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) ->
-    gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}).
-
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
-start_link() ->
-    gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []).
-
-%%%===================================================================
-%%% gen_statem callbacks
-%%%===================================================================
-
--spec init(list()) -> {ok, atom(), #state{}}.
-init([]) ->
-    case efka_iot_outbox:open(outbox_options()) of
-        {ok, Outbox} ->
-            erlang:start_timer(0, self(), create_transport),
-            {ok, ?STATE_DISCONNECTED, #state{socket = undefined, outbox = Outbox}};
-        {error, Reason} ->
-            {stop, Reason}
-    end.
-
--spec callback_mode() -> handle_event_function.
-callback_mode() ->
-    handle_event_function.
-
--spec outbox_options() -> map().
-outbox_options() ->
-    {ok, DetsDir} = application:get_env(efka, dets_dir),
-    #{
-        dir => filename:join(DetsDir, "iot_outbox"),
-        segment_record_limit => ?OUTBOX_SEGMENT_RECORD_LIMIT,
-        max_segments => ?OUTBOX_MAX_SEGMENTS,
-        max_record_bytes => ?OUTBOX_MAX_RECORD_BYTES
-    }.
-
-%% 异步发送数据，连接存在时直接发送；否则写入持久化 outbox。
--spec handle_event(term(), term(), atom(), #state{}) -> term().
-handle_event(cast, {metric_data, RouteKey, Metric}, StateName, State = #state{socket = Socket}) ->
-    Packet = encode_message_frame({metric_data, RouteKey, Metric}),
-    case StateName of
-        ?STATE_ACTIVATED ->
-            ok = ssl:send(Socket, Packet),
-            {keep_state, State};
-        _ ->
-            case efka_iot_outbox:append(Packet, State#state.outbox) of
-                {ok, Outbox} ->
-                    {keep_state, State#state{outbox = Outbox}};
-                {dropped, capacity_reached, Outbox} ->
-                    logger:warning("[efka_iot_client] outbox capacity reached, drop offline metric"),
-                    {keep_state, State#state{
-                        outbox = Outbox,
-                        dropped_message_count = State#state.dropped_message_count + 1
-                    }};
-                {error, Reason} ->
-                    logger:warning("[efka_iot_client] append outbox failed, reason: ~p", [Reason]),
-                    {keep_state, State#state{dropped_message_count = State#state.dropped_message_count + 1}}
-            end
-    end;
-
-handle_event(cast, {send_stream, StreamId, Body}, ?STATE_ACTIVATED, State = #state{socket = Socket})
-    when is_integer(StreamId), StreamId > 0 ->
-    Packet = encode_stream_frame(StreamId, Body),
-    ok = ssl:send(Socket, Packet),
-    {keep_state, State};
-handle_event(cast, {send_stream, _StreamId, _Body}, _StateName, State) ->
-    {keep_state, State};
-
-handle_event(cast, {stream_done, StreamId}, _StateName, State = #state{streams = Streams})
-    when is_integer(StreamId), StreamId > 0 ->
-    case maps:take(StreamId, Streams) of
-        error ->
-            {keep_state, State};
-        {StreamState, NStreams} ->
-            demonitor_stream(StreamState),
-            {keep_state, State#state{streams = NStreams}}
-    end;
-
-%% 其他情况下直接忽略
-handle_event(cast, _, _, State = #state{}) ->
-    {keep_state, State};
-
-handle_event({call, From}, is_activated, ?STATE_ACTIVATED, State = #state{}) ->
-    {keep_state, State, [{reply, From, true}]};
-handle_event({call, From}, is_activated, _StateName, State = #state{}) ->
-    {keep_state, State, [{reply, From, false}]};
-handle_event({call, From}, dropped_message_count, _StateName, State = #state{dropped_message_count = DroppedCount}) ->
-    {keep_state, State, [{reply, From, DroppedCount}]};
-
-%% 异步建立到服务器的连接
-handle_event(info, {timeout, _, create_transport}, ?STATE_DISCONNECTED, State = #state{next_pkt_id = PktId}) ->
-    case connect_socket() of
-        {ok, Socket} ->
-            AuthPacket = auth_packet(PktId),
-            ok = ssl:send(Socket, AuthPacket),
-            logger:debug("[efka_iot_client] send auth request, packet_id: ~p", [PktId]),
-            {next_state, ?STATE_AUTH, State#state{socket = Socket, auth_pkt_id = PktId, next_pkt_id = PktId + 1}, [{state_timeout, 5000, auth_timeout}]};
-        {error, _Reason} ->
-            schedule_reconnect(),
-            {keep_state, State#state{socket = undefined}}
-    end;
-
-handle_event(state_timeout, auth_timeout, ?STATE_AUTH, State = #state{socket = Socket}) ->
-    logger:debug("[efka_iot_client] auth request timeout"),
-    disconnect(Socket),
-    schedule_reconnect(),
-    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_pkt_id = undefined}};
-
-handle_event(info, {timeout, TimerRef, ssl_ping}, ?STATE_ACTIVATED, State = #state{socket = Socket, ping_timer_ref = TimerRef}) ->
-    Packet = encode_message_frame(ping),
-    case ssl:send(Socket, Packet) of
-        ok ->
-            {keep_state, schedule_ssl_ping(State)};
-        {error, Reason} ->
-            logger:warning("[efka_iot_client] send ssl ping failed, reason: ~p", [Reason]),
-            disconnect(Socket),
-            schedule_reconnect(),
-            NState = close_all_streams({send_ping_failed, Reason}, State),
-            {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}}
-    end;
-handle_event(info, {timeout, _TimerRef, ssl_ping}, _StateName, State) ->
-    {keep_state, State};
-
-%% 将缓存中的数据推送到服务器端
-handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
-    case efka_iot_outbox:next(State#state.outbox) of
-        {ok, Seq, Packet} ->
-            ok = ssl:send(Socket, Packet),
-            case efka_iot_outbox:ack(Seq, State#state.outbox) of
-                {ok, Outbox} ->
-                    {keep_state, State#state{outbox = Outbox}, [{next_event, info, flush_cache}]};
-                {error, Reason} ->
-                    logger:warning("[efka_iot_client] ack outbox failed, seq: ~p, reason: ~p", [Seq, Reason]),
-                    {keep_state, State}
-            end;
-        eof ->
-            {keep_state, State};
-        {error, Reason} ->
-            logger:warning("[efka_iot_client] read outbox failed, reason: ~p", [Reason]),
-            {keep_state, State}
-    end;
-handle_event(info, flush_cache, _, State) ->
-    {keep_state, State};
-
-%% 处理收到的ssl消息
-handle_event(info, {ssl, Socket, PacketBin}, _, State = #state{socket = Socket}) when is_binary(PacketBin) ->
-    case decode_frame(PacketBin) of
-        {ok, Packet} ->
-            {keep_state, State, [{next_event, internal, Packet}]};
-        {error, Reason} ->
-            logger:warning("[efka_iot_client] decode packet failed: ~p, packet_size: ~p", [Reason, byte_size(PacketBin)]),
-            disconnect(Socket),
-            cancel_ssl_ping(State),
-            schedule_reconnect(),
-            NState = close_all_streams(bad_packet, State),
-            {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}}
-    end;
-handle_event(info, {ssl_error, Socket, Reason}, _, State = #state{}) ->
-    logger:debug("[efka_iot_client] ssl error: ~p", [Reason]),
-    disconnect(Socket),
-    cancel_ssl_ping(State),
-    schedule_reconnect(),
-    NState = close_all_streams(ssl_error, State),
-    {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}};
-handle_event(info, {ssl_closed, Socket}, _, State = #state{}) ->
-    logger:debug("[efka_iot_client] ssl closed"),
-    disconnect(Socket),
-    cancel_ssl_ping(State),
-    schedule_reconnect(),
-    NState = close_all_streams(ssl_closed, State),
-    {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}};
-
-handle_event(info, {'DOWN', MonitorRef, process, WorkerPid, Reason}, _StateName, State = #state{streams = Streams}) ->
-    case take_stream_by_monitor(MonitorRef, WorkerPid, Streams) of
-        error ->
-            {keep_state, State};
-        {StreamId, StreamState, NStreams} ->
-            demonitor_stream(StreamState),
-            case Reason of
-                normal ->
-                    {keep_state, State#state{streams = NStreams}};
-                _ ->
-                    logger:warning("[efka_iot_client] stream worker down, stream_id: ~p, reason: ~p", [StreamId, Reason]),
-                    case State#state.socket of
-                        undefined ->
-                            {keep_state, State#state{streams = NStreams}};
-                        Socket ->
-                            Packet = encode_stream_frame(StreamId, {reset, {worker_down, Reason}}),
-                            ok = ssl:send(Socket, Packet),
-                            {keep_state, State#state{streams = NStreams}}
-                    end
-            end
-    end;
-
-%%% 处理内部消息。TLS 收到的数据先经过 protobuf 解码，再由这里按本地事件分发。
-
-%% 容器管理命令由 iot 发起，使用 command/command_response 语义。
-handle_event(internal, #'Command'{packet_id = PktId, body = {container, ContainerCommand}}, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
-    handle_container_command(PktId, ContainerCommand, Socket),
-    {keep_state, State};
-handle_event(internal, #'Command'{packet_id = PktId, body = {container, ContainerCommand}}, _StateName, State = #state{socket = Socket}) ->
-    logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [ContainerCommand]),
-    send_container_response(Socket, PktId, {error, <<"agent invalid">>}),
-    {keep_state, State};
-
-%% 处理response
-handle_event(internal, #'Response'{packet_id = AuthPktId, body = {auth_response, #'Response.AuthResponse'{}}}, ?STATE_AUTH, State = #state{auth_pkt_id = AuthPktId}) ->
-    logger:debug("[efka_iot_client] auth success"),
-    State1 = schedule_ssl_ping(State#state{auth_pkt_id = undefined}),
-    {next_state, ?STATE_ACTIVATED, State1, [{next_event, info, flush_cache}]};
-handle_event(internal, #'Response'{packet_id = AuthPktId, body = {error, #'Response.Error'{reason = Reason}}}, ?STATE_AUTH, State = #state{socket = Socket, auth_pkt_id = AuthPktId}) ->
-    logger:debug("[efka_iot_client] auth failed, reason: ~p", [Reason]),
-    disconnect(Socket),
-    schedule_reconnect(),
-    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_pkt_id = undefined}};
-handle_event(internal, #'Response'{} = Reply, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore unexpected response in state ~p: ~p", [StateName, Reply]),
-    {keep_state, State};
-handle_event(internal, #'CommandResponse'{} = Reply, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore unexpected command_response in state ~p: ~p", [StateName, Reply]),
-    {keep_state, State};
-
-%% 透明 TCP stream 多路复用。
-handle_event(internal, #'Stream'{stream_id = StreamId, payload = Payload}, ?STATE_ACTIVATED, State) ->
-    Body = case Payload of
-               {open, #'Stream.Open'{target = Target}} ->
-                   {open, Target};
-               {opened, #'Stream.Opened'{}} ->
-                   opened;
-               {open_error, #'Stream.OpenError'{reason = Reason}} ->
-                   {open_error, Reason};
-               {data, #'Stream.Data'{bytes = Data}} ->
-                   {data, Data};
-               {fin, #'Stream.Fin'{}} ->
-                   fin;
-               {reset, #'Stream.Reset'{reason = Reason}} ->
-                   {reset, Reason}
-           end,
-    handle_stream_frame(StreamId, Body, State);
-handle_event(internal, #'Stream'{stream_id = StreamId, payload = Payload}, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore stream frame in state ~p, stream_id: ~p, body: ~p",
-        [StateName, StreamId, Payload]),
-    {keep_state, State};
-
-%% 处理Pub/Sub机制
-handle_event(internal, #'Message'{body = {pong, #'Message.Pong'{}}}, ?STATE_ACTIVATED, State) ->
-    {keep_state, State};
-handle_event(internal, #'Message'{body = {pub, #'Message.Pub'{topic = Topic, qos = Qos, content = Content}}}, ?STATE_ACTIVATED, State) ->
-    logger:debug("[efka_iot_client] get pub topic: ~p, qos: ~p, content: ~p", [Topic, Qos, Content]),
-    efka_subscription:publish(Topic, Qos, Content),
-    {keep_state, State};
-handle_event(internal, Packet, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore unknown packet: ~p, state_name: ~p", [Packet, StateName]),
-    {keep_state, State};
-
-handle_event(info, Info, _, State = #state{}) ->
-    logger:notice("[efka_iot_client] get unknown info: ~p", [Info]),
-    {keep_state, State}.
-
--spec terminate(term(), atom(), #state{}) -> ok.
-terminate(Reason, _StateName, State = #state{socket = Socket, outbox = Outbox}) ->
-    cancel_ssl_ping(State),
-    _ = close_all_streams(Reason, State),
-    disconnect(Socket),
-    efka_iot_outbox:close(Outbox),
-    logger:notice("[efka_iot_client] terminate with reason: ~p", [Reason]),
-    ok.
-
--spec code_change(term(), atom(), #state{}, term()) -> {ok, atom(), #state{}}.
-code_change(_OldVsn, StateName, State = #state{}, _Extra) ->
-    {ok, StateName, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec auth_packet(pos_integer()) -> binary().
-auth_packet(PktId) when is_integer(PktId), PktId > 0 ->
-    {ok, AuthInfo} = application:get_env(efka, auth),
-    UUID = proplists:get_value(uuid, AuthInfo),
-    Token = proplists:get_value(token, AuthInfo),
-
-    Timestamp = efka_util:timestamp(),
-    encode_transport_frame(?CLASS_REQUEST, #'Request'{
-        packet_id = PktId,
-        body = {auth_request, #'Request.AuthRequest'{
-            uuid = list_to_binary(UUID),
-            token = list_to_binary(Token),
-            timestamp = Timestamp
-        }}
-    }).
-
--spec encode_message_frame(term()) -> binary().
-encode_message_frame(ping) ->
-    encode_transport_frame(?CLASS_MESSAGE, #'Message'{body = {ping, #'Message.Ping'{}}});
-encode_message_frame({metric_data, RouteKey, Metric}) ->
-    encode_transport_frame(?CLASS_MESSAGE, #'Message'{
-        body = {metric_data, #'Message.MetricData'{route_key = RouteKey, metric = Metric}}
-    });
-encode_message_frame(Body) ->
-    error({unsupported_message_body, Body}).
-
--spec encode_stream_frame(stream_id(), term()) -> binary().
-encode_stream_frame(StreamId, Body) ->
-    Payload = case Body of
-                  open ->
-                      {open, #'Stream.Open'{target = ?STREAM_TARGET_MANAGER}};
-                  {open, Target} when is_integer(Target), Target > 0 ->
-                      {open, #'Stream.Open'{target = Target}};
-                  opened ->
-                      {opened, #'Stream.Opened'{}};
-                  {open_error, Reason} ->
-                      {open_error, #'Stream.OpenError'{reason = reason_to_binary(Reason)}};
-                  {data, Data} when is_binary(Data) ->
-                      {data, #'Stream.Data'{bytes = Data}};
-                  fin ->
-                      {fin, #'Stream.Fin'{}};
-                  {reset, Reason} ->
-                      {reset, #'Stream.Reset'{reason = reason_to_binary(Reason)}}
-              end,
-    encode_transport_frame(?CLASS_STREAM, #'Stream'{stream_id = StreamId, payload = Payload}).
-
--spec encode_transport_frame(byte(), message_pb:'$msg'()) -> binary().
-encode_transport_frame(Class, Msg) ->
-    Payload = message_pb:encode_msg(Msg),
-    <<Class, Payload/binary>>.
-
--spec decode_frame(binary()) -> {ok, tuple()} | {error, term()}.
-decode_frame(<<?CLASS_RESPONSE, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Response');
-decode_frame(<<?CLASS_COMMAND, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Command');
-decode_frame(<<?CLASS_COMMAND_RESPONSE, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'CommandResponse');
-decode_frame(<<?CLASS_MESSAGE, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Message');
-decode_frame(<<?CLASS_STREAM, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Stream');
-decode_frame(<<?CLASS_REQUEST, _Payload/binary>>) ->
-    {error, unsupported_request_frame};
-decode_frame(Other) ->
-    {error, {invalid_frame, Other}}.
-
--spec decode_pb_frame(binary(), message_pb:'$msg_name'()) ->
-    {ok, tuple()} | {error, term()}.
-decode_pb_frame(Payload, MsgName) ->
-    try message_pb:decode_msg(Payload, MsgName) of
-        Msg ->
-            {ok, Msg}
-    catch
-        Class:Reason ->
-            {error, {bad_protobuf, MsgName, Class, Reason}}
-    end.
-
--spec connect_socket() -> {ok, ssl:sslsocket()} | {error, term()}.
-connect_socket() ->
-    {ok, Props} = application:get_env(efka, iot_server),
-    Host = proplists:get_value(host, Props),
-    Port = proplists:get_value(tls_port, Props),
-    SslOptions = [
-        binary,
-        {active, true},
-        {packet, 4},
-        {verify, verify_none}
-    ],
-    ssl:connect(Host, Port, SslOptions, 5000).
-
--spec disconnect(undefined | ssl:sslsocket()) -> ok.
-disconnect(undefined) ->
-    ok;
-disconnect(Socket) ->
-    catch ssl:close(Socket),
-    ok.
-
--spec schedule_reconnect() -> reference().
-schedule_reconnect() ->
-    erlang:start_timer(5000, self(), create_transport).
-
--spec schedule_ssl_ping(#state{}) -> #state{}.
-schedule_ssl_ping(State = #state{ping_timer_ref = TimerRef}) ->
-    cancel_timer(TimerRef),
-    State#state{ping_timer_ref = erlang:start_timer(?SSL_PING_INTERVAL, self(), ssl_ping)}.
-
--spec cancel_ssl_ping(#state{}) -> ok.
-cancel_ssl_ping(#state{ping_timer_ref = TimerRef}) ->
-    cancel_timer(TimerRef).
-
--spec cancel_timer(undefined | reference()) -> ok.
-cancel_timer(undefined) ->
-    ok;
-cancel_timer(TimerRef) ->
-    _ = erlang:cancel_timer(TimerRef),
-    ok.
-
--spec send_container_response(ssl:sslsocket(), pos_integer(), term()) -> ok.
-send_container_response(Socket, PktId, Reply) ->
-    Body = case Reply of
-               ok ->
-                   {result, <<"ok">>};
-               {ok, Result} ->
-                   {result, reply_to_binary(Result)};
-               {error, Reason} ->
-                   {error, #'CommandResponse.Error'{code = 1, reason = reason_to_binary(Reason)}};
-               Other ->
-                   {result, reply_to_binary(Other)}
-           end,
-    Packet = encode_transport_frame(?CLASS_COMMAND_RESPONSE, #'CommandResponse'{
-        packet_id = PktId,
-        body = Body
-    }),
-    ok = ssl:send(Socket, Packet).
-
--spec handle_container_command(pos_integer(), message_pb:'Command.Container'(), ssl:sslsocket()) -> ok.
-handle_container_command(PktId, #'Command.Container'{action = {list, #'Command.Container.ContainerList'{}}}, Socket) ->
-    Reply = docker_commands:get_containers(),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {start, #'Command.Container.ContainerStart'{target = Target}}}, Socket) ->
-    Reply = docker_commands:start_container(container_target(Target)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {stop, #'Command.Container.ContainerStop'{target = Target, timeout_seconds = TimeoutSeconds}}}, Socket) ->
-    Reply = docker_commands:stop_container(container_target(Target), TimeoutSeconds),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {kill, #'Command.Container.ContainerKill'{target = Target, signal = Signal}}}, Socket) ->
-    Reply = docker_commands:kill_container(container_target(Target), to_binary(Signal)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {remove, #'Command.Container.ContainerRemove'{target = Target, force = Force, remove_volumes = RemoveVolumes}}}, Socket) ->
-    Reply = docker_commands:remove_container(container_target(Target), to_bool(Force), to_bool(RemoveVolumes)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {config, #'Command.Container.ContainerConfig'{target = Target, config = Config}}}, Socket) ->
-    Reply = docker_helper:update_container_config(container_target(Target), iolist_to_binary(Config)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, ContainerCommand, Socket) ->
-    logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [ContainerCommand]),
-    send_container_response(Socket, PktId, {error, <<"agent invalid">>}),
-    ok.
-
-%% iot 发起 open 时携带 target；efka 根据本地 stream_targets 配置建连。
--spec handle_stream_frame(term(), term(), #state{}) -> gen_statem:event_handler_result(atom(), #state{}).
-handle_stream_frame(StreamId, {open, Target}, State = #state{streams = Streams})
-    when is_integer(StreamId), StreamId > 0, is_integer(Target), Target > 0 ->
-    case valid_iot_stream_id(StreamId) andalso not maps:is_key(StreamId, Streams) of
-        true ->
-            case start_stream_worker(StreamId, Target) of
-                {ok, {WorkerPid, MonitorRef}} ->
-                    StreamState = #stream_state{worker_pid = WorkerPid, monitor_ref = MonitorRef},
-                    {keep_state, State#state{streams = maps:put(StreamId, StreamState, Streams)}};
-                {error, Reason} ->
-                    send_stream(StreamId, {open_error, Reason}),
-                    {keep_state, State}
-            end;
-        false ->
-            send_stream(StreamId, {reset, <<"invalid stream open">>}),
-            {keep_state, State}
-    end;
-handle_stream_frame(StreamId, {open, Target}, State)
-    when is_integer(StreamId), StreamId > 0 ->
-    logger:warning("[efka_iot_client] invalid stream target, stream_id: ~p, target: ~p", [StreamId, Target]),
-    send_stream(StreamId, {open_error, <<"invalid stream target">>}),
-    {keep_state, State};
-handle_stream_frame(StreamId, Body, State = #state{streams = Streams})
-    when is_integer(StreamId), StreamId > 0 ->
-    case maps:get(StreamId, Streams, undefined) of
-        undefined ->
-            maybe_reset_unknown_stream(StreamId, Body),
-            {keep_state, State};
-        StreamState = #stream_state{worker_pid = WorkerPid} ->
-            WorkerPid ! {stream, StreamId, Body},
-            case Body of
-                {reset, _Reason} ->
-                    demonitor_stream(StreamState),
-                    {keep_state, State#state{streams = maps:remove(StreamId, Streams)}};
-                _ ->
-                    {keep_state, State}
-            end
-    end;
-handle_stream_frame(StreamId, Body, State) ->
-    logger:warning("[efka_iot_client] invalid stream frame, stream_id: ~p, body: ~p", [StreamId, Body]),
-    {keep_state, State}.
-
--spec start_stream_worker(stream_id(), pos_integer()) -> {ok, {pid(), reference()}} | {error, term()}.
-start_stream_worker(StreamId, ?STREAM_TARGET_MANAGER) ->
-    efka_iot_stream:start_stream(StreamId, ?STREAM_TARGET_MANAGER);
-start_stream_worker(StreamId, ?STREAM_TARGET_CONTAINER_DEPLOY) ->
-    efka_iot_deploy_stream:start_stream(StreamId);
-start_stream_worker(_StreamId, Target) ->
-    {error, {unknown_stream_target, Target}}.
-
--spec maybe_reset_unknown_stream(stream_id(), term()) -> ok.
-maybe_reset_unknown_stream(_StreamId, {reset, _Reason}) ->
-    ok;
-maybe_reset_unknown_stream(_StreamId, fin) ->
-    ok;
-maybe_reset_unknown_stream(_StreamId, {open_error, _Reason}) ->
-    ok;
-maybe_reset_unknown_stream(StreamId, _Body) ->
-    send_stream(StreamId, {reset, <<"unknown stream">>}).
-
--spec valid_iot_stream_id(stream_id()) -> boolean().
-valid_iot_stream_id(StreamId) ->
-    StreamId rem 2 =:= 1.
-
--spec take_stream_by_monitor(reference(), pid(), map()) ->
-    {stream_id(), #stream_state{}, map()} | error.
-take_stream_by_monitor(MonitorRef, WorkerPid, Streams) ->
-    take_stream_by_monitor(MonitorRef, WorkerPid, maps:iterator(Streams), Streams).
-
-take_stream_by_monitor(MonitorRef, WorkerPid, Iter, Streams) ->
-    case maps:next(Iter) of
-        none ->
-            error;
-        {StreamId, StreamState = #stream_state{worker_pid = WorkerPid, monitor_ref = MonitorRef}, _NextIter} ->
-            {StreamId, StreamState, maps:remove(StreamId, Streams)};
-        {_StreamId, _StreamState, NextIter} ->
-            take_stream_by_monitor(MonitorRef, WorkerPid, NextIter, Streams)
-    end.
-
--spec close_all_streams(term(), #state{}) -> #state{}.
-close_all_streams(Reason, State = #state{streams = Streams}) ->
-    maps:foreach(fun(StreamId, StreamState = #stream_state{worker_pid = WorkerPid}) ->
-        demonitor_stream(StreamState),
-        WorkerPid ! {stream, StreamId, {reset, {channel_closed, Reason}}}
-    end, Streams),
-    State#state{streams = #{}}.
-
--spec demonitor_stream(#stream_state{}) -> ok.
-demonitor_stream(#stream_state{monitor_ref = MonitorRef}) ->
-    erlang:demonitor(MonitorRef, [flush]),
-    ok.
-
--spec reply_to_binary(term()) -> binary().
-reply_to_binary(Value) when is_binary(Value) ->
-    Value;
-reply_to_binary(Value) ->
-    try iolist_to_binary(Value) of
-        Bin ->
-            Bin
-    catch
-        _:_ ->
-            try iolist_to_binary(json:encode(Value)) of
-                JsonBin ->
-                    JsonBin
-            catch
-                _:_ ->
-                    unicode:characters_to_binary(io_lib:format("~p", [Value]))
-            end
-    end.
-
--spec reason_to_binary(term()) -> binary().
-reason_to_binary(Reason) when is_binary(Reason) ->
-    Reason;
-reason_to_binary(Reason) ->
-    try iolist_to_binary(Reason) of
-        Bin ->
-            Bin
-    catch
-        _:_ ->
-            unicode:characters_to_binary(io_lib:format("~p", [Reason]))
-    end.
-
--spec container_target(message_pb:'Command.Container.ContainerTarget'() | undefined) -> binary().
-container_target(#'Command.Container.ContainerTarget'{name = Name, id = Id}) ->
-    NameBin = iolist_to_binary(Name),
-    IdBin = iolist_to_binary(Id),
-    case NameBin of
-        <<>> ->
-            true = IdBin =/= <<>>,
-            IdBin;
-        _ ->
-            NameBin
-    end;
-container_target(undefined) ->
-    error(bad_container_target).
-
--spec to_binary(binary() | list()) -> binary().
-to_binary(Value) when is_binary(Value) ->
-    Value;
-to_binary(Value) when is_list(Value) ->
-    unicode:characters_to_binary(Value).
-
--spec to_bool(true | false | 0 | 1) -> boolean().
-to_bool(true) ->
-    true;
-to_bool(1) ->
-    true;
-to_bool(false) ->
-    false;
-to_bool(0) ->
-    false.

apps/efka/src/iot/efka_iot_deploy_stream.erl

@@ -1,119 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc One container deploy feedback stream from iot.
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_deploy_stream).
-
--export([start_stream/1]).
--export([run/1]).
-
--define(REQUEST_TIMEOUT, 10000).
--define(MAX_REQUEST_BYTES, 16 * 1024 * 1024).
-
--type stream_id() :: pos_integer().
-
--spec start_stream(StreamId :: stream_id()) -> {ok, {pid(), reference()}}.
-start_stream(StreamId) when is_integer(StreamId), StreamId > 0 ->
-    {ok, spawn_monitor(?MODULE, run, [StreamId])}.
-
--spec run(StreamId :: stream_id()) -> ok.
-run(StreamId) when is_integer(StreamId), StreamId > 0 ->
-    try run0(StreamId) of
-        ok ->
-            ok
-    catch
-        Class:Reason:Stack ->
-            logger:warning("[efka_iot_deploy_stream] stream_id: ~p crashed, class: ~p, reason: ~p, stack: ~p",
-                [StreamId, Class, Reason, Stack]),
-            send_error_and_close(StreamId, iolist_to_binary(io_lib:format("~p:~p", [Class, Reason]))),
-            ok
-    after
-        efka_iot_client:stream_done(StreamId)
-    end.
-
--spec run0(stream_id()) -> ok.
-run0(StreamId) ->
-    efka_iot_client:send_stream(StreamId, opened),
-    case receive_request(StreamId, <<>>) of
-        {ok, Request} ->
-            handle_request(StreamId, Request);
-        {error, reset} ->
-            ok;
-        {error, Reason} ->
-            send_error_and_close(StreamId, reason_to_binary(Reason))
-    end.
-
--spec receive_request(stream_id(), binary()) -> {ok, binary()} | {error, term()}.
-receive_request(StreamId, Acc) ->
-    receive
-        {stream, StreamId, {data, Data}} when is_binary(Data) ->
-            NAcc = <<Acc/binary, Data/binary>>,
-            case byte_size(NAcc) =< ?MAX_REQUEST_BYTES of
-                true ->
-                    receive_request(StreamId, NAcc);
-                false ->
-                    {error, request_too_large}
-            end;
-        {stream, StreamId, fin} ->
-            {ok, Acc};
-        {stream, StreamId, {reset, _Reason}} ->
-            {error, reset};
-        Info ->
-            logger:debug("[efka_iot_deploy_stream] stream_id: ~p ignore unknown info: ~p", [StreamId, Info]),
-            receive_request(StreamId, Acc)
-    after ?REQUEST_TIMEOUT ->
-        {error, request_timeout}
-    end.
-
--spec handle_request(stream_id(), binary()) -> ok.
-handle_request(StreamId, RequestBin) ->
-    case decode_request(RequestBin) of
-        {ok, TaskId, Params} ->
-            deploy(StreamId, TaskId, Params);
-        {error, Reason} ->
-            send_error_and_close(StreamId, reason_to_binary(Reason))
-    end.
-
--spec decode_request(binary()) -> {ok, non_neg_integer(), map()} | {error, term()}.
-decode_request(RequestBin) ->
-    try json:decode(RequestBin) of
-        #{<<"task_id">> := TaskId, <<"params">> := Params}
-          when is_integer(TaskId), TaskId >= 0, is_map(Params) ->
-            {ok, TaskId, Params};
-        _ ->
-            {error, invalid_deploy_request}
-    catch
-        Class:Reason ->
-            {error, {bad_json, Class, Reason}}
-    end.
-
--spec deploy(stream_id(), non_neg_integer(), map()) -> ok.
-deploy(StreamId, TaskId, Params) ->
-    try
-        ContainerName = maps:get(<<"container_name">>, Params),
-        {ok, RootDir} = docker_helper:root_dir(),
-        {ok, ContainerDir} = docker_helper:ensure_container_dir(RootDir, ContainerName),
-        docker_deployer:deploy(TaskId, ContainerDir, Params, {stream, StreamId})
-    catch
-        Class:Reason ->
-            Error = iolist_to_binary(io_lib:format("deploy stream failed: ~p:~p", [Class, Reason])),
-            send_error_and_close(StreamId, Error)
-    end.
-
--spec send_error_and_close(stream_id(), binary()) -> ok.
-send_error_and_close(StreamId, Reason) ->
-    ok = efka_iot_client:send_stream(StreamId, {data, iolist_to_binary(json:encode(#{
-        <<"type">> => <<"error">>,
-        <<"message">> => Reason
-    }))}),
-    ok = efka_iot_client:send_stream(StreamId, {data, iolist_to_binary(json:encode(#{
-        <<"type">> => <<"close">>,
-        <<"reason">> => <<"fail">>
-    }))}),
-    efka_iot_client:send_stream(StreamId, fin).
-
--spec reason_to_binary(term()) -> binary().
-reason_to_binary(Reason) when is_binary(Reason) ->
-    Reason;
-reason_to_binary(Reason) ->
-    unicode:characters_to_binary(io_lib:format("~p", [Reason])).

apps/efka/src/iot/efka_iot_heartbeat.erl

@@ -1,119 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc UDP heartbeat sender for iot host liveness.
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_heartbeat).
-
--behaviour(gen_server).
-
-%% API
--export([start_link/0]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE).
--define(HEARTBEAT_VERSION, 1).
--define(HEARTBEAT_NONCE_BYTES, 16).
--define(DEFAULT_INTERVAL, 5000).
-
--record(state, {
-    socket :: gen_udp:socket(),
-    host :: inet:hostname() | inet:ip_address(),
-    port :: inet:port_number(),
-    interval :: pos_integer(),
-    uuid :: binary(),
-    heartbeat_secret :: binary()
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
-start_link() ->
-    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
--spec init(list()) -> {ok, #state{}} | {stop, term()}.
-init([]) ->
-    ok = application:ensure_started(crypto),
-    {ok, ServerProps} = application:get_env(efka, iot_server),
-    {ok, AuthProps} = application:get_env(efka, auth),
-    HeartbeatProps = case application:get_env(efka, heartbeat) of
-        {ok, Props} -> Props;
-        undefined -> []
-    end,
-    Host = proplists:get_value(host, ServerProps),
-    UdpPort = proplists:get_value(udp_port, ServerProps),
-    Interval = proplists:get_value(interval, HeartbeatProps, ?DEFAULT_INTERVAL),
-    UUID = list_to_binary(proplists:get_value(uuid, AuthProps)),
-    Token = list_to_binary(proplists:get_value(token, AuthProps)),
-    HeartbeatSecret = crypto:hash(sha256, Token),
-    case gen_udp:open(0, [binary]) of
-        {ok, Socket} ->
-            erlang:send_after(0, self(), heartbeat),
-            {ok, #state{
-                socket = Socket,
-                host = Host,
-                port = UdpPort,
-                interval = Interval,
-                uuid = UUID,
-                heartbeat_secret = HeartbeatSecret
-            }};
-        {error, Reason} ->
-            {stop, Reason}
-    end.
-
--spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, ok, #state{}}.
-handle_call(_Request, _From, State) ->
-    {reply, ok, State}.
-
--spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
-handle_cast(_Request, State) ->
-    {noreply, State}.
-
--spec handle_info(term(), #state{}) -> {noreply, #state{}}.
-handle_info(heartbeat, State = #state{interval = Interval}) ->
-    ok = send_heartbeat(State),
-    erlang:send_after(Interval, self(), heartbeat),
-    {noreply, State};
-handle_info(Info, State) ->
-    logger:warning("[efka_iot_heartbeat] ignore unknown info: ~p", [Info]),
-    {noreply, State}.
-
--spec terminate(term(), #state{}) -> ok.
-terminate(_Reason, #state{socket = Socket}) ->
-    gen_udp:close(Socket),
-    ok.
-
--spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
-code_change(_OldVsn, State, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec send_heartbeat(#state{}) -> ok.
-send_heartbeat(#state{socket = Socket, host = Host, port = Port, uuid = UUID, heartbeat_secret = HeartbeatSecret}) ->
-    Packet = heartbeat_packet_iodata(UUID, HeartbeatSecret),
-    case gen_udp:send(Socket, Host, Port, Packet) of
-        ok ->
-            ok;
-        {error, Reason} ->
-            logger:warning("[efka_iot_heartbeat] send heartbeat failed, reason: ~p", [Reason]),
-            ok
-    end.
-
--spec heartbeat_packet_iodata(binary(), binary()) -> iodata().
-heartbeat_packet_iodata(UUID, HeartbeatSecret) when is_binary(UUID), is_binary(HeartbeatSecret) ->
-    UUIDLen = byte_size(UUID),
-    Timestamp = efka_util:timestamp(),
-    Nonce = crypto:strong_rand_bytes(?HEARTBEAT_NONCE_BYTES),
-    Payload = <<?HEARTBEAT_VERSION:8, UUIDLen:16, UUID:UUIDLen/binary, Timestamp:64/unsigned-big, Nonce/binary>>,
-    Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload),
-    [Payload, Mac].

apps/efka/src/iot/efka_iot_outbox.erl

@@ -1,662 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc
-%%% efka_iot_client 的持久化待发送队列。
-%%%
-%%% 本模块用于保存 efka_iot_client 离线期间产生的待发送 packet。底层使用
-%%% 追加写 segment 文件的方式实现。当前场景只有一个写入方和一个读取方，并且
-%%% 读取方每次只读取一条数据。
-%%%
-%%% 文件结构：
-%%% - `segment-N.log'：一个 segment 文件，按写入顺序追加保存 record。
-%%% - `metadata.json'：使用原始 JSON 文本保存 write_seq、acked_seq 和
-%%%   writer_segment。进程启动时仍然会扫描 segment 文件，并以 segment 文件
-%%%   里的真实数据作为准确信息；因此即使 metadata 落后，也不会导致跳过或删除
-%%%   未消费的数据。
-%%% - outbox 目录通过 open/1 的 dir 参数传入，本模块不直接读取应用环境变量。
-%%%
-%%% open/1 参数：
-%%% - dir：outbox 数据目录。
-%%% - segment_record_limit：每个 segment 最多保存的 record 数。
-%%% - max_segments：最多保留的 live segment 数。
-%%% - max_record_bytes：允许读取的单条 record 最大 payload 字节数。
-%%%
-%%% record 格式：
-%%% - 8 字节 magic bytes，当前为 <<"EFKAOBX1">>。
-%%% - 1 字节 version，当前为 1。
-%%% - 1 字节 header size，当前为 22。
-%%% - 4 字节 unsigned big-endian packet size。
-%%% - 8 字节 unsigned big-endian Seq。
-%%% - Packet 原始二进制内容。
-%%% - 4 字节 unsigned big-endian crc32，校验范围为 header + Packet。
-%%% - 读取时会先搜索 magic，再校验 version、header size、packet size 和
-%%%   crc32。遇到损坏 record 时，会继续向后寻找下一个合法 magic，尽量恢复
-%%%   后续可读数据。
-%%%
-%%% 写入逻辑：
-%%% - `append/2' 将一条 record 写入当前 writer segment，随后 fsync 当前
-%%%   segment，推进 write_seq，并持久化 metadata。
-%%% - 每个 segment 最多保存多少条数据由 open/1 的 segment_record_limit
-%%%   参数决定。
-%%% - outbox 最多保留多少个 live segment 由 open/1 的 max_segments 参数
-%%%   决定。当当前 segment 已满，且 live segment 数已经达到上限时，新
-%%%   packet 会被丢弃，已有磁盘数据保持不变。
-%%% - 当当前 segment 已满，但 live segment 数量尚未达到上限时，writer 会
-%%%   关闭旧文件，并开始写入 `segment-(N + 1).log'。
-%%%
-%%% 读取逻辑：
-%%% - `next/1' 找到第一个 end_seq 大于 acked_seq 的 live segment，打开该
-%%%   segment，并只返回第一条 Seq 大于 acked_seq 的 record。
-%%% - 本模块不做批量读取；efka_iot_client 每次只发送并 ack 一条 packet。
-%%% - `ack/2' 接受当前读取到的 Seq。正常情况下 Seq 连续递增；如果某条
-%%%   record 损坏并被读取端跳过，ack 允许推进到后续合法 record 的 Seq，
-%%%   避免单条坏数据阻断整个 segment 后续数据的消费。
-%%%
-%%% segment 清理逻辑：
-%%% - 当 `ack/2' 消费完某个 segment 的最后一条 record，并且后续 segment
-%%%   里还有未读数据时，已消费的 segment 文件会被删除；下一次读取会从后续
-%%%   segment 开始。
-%%% - 当 acked_seq 追上 write_seq 时，说明所有 record 都已经投递完成；此时
-%%%   会删除所有 segment 文件，将计数器重置为 0，下一次 append 从全新的
-%%%   `segment-1.log' 的 Seq 1 开始。
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_outbox).
-
--export([open/1, close/1, append/2, next/1, ack/2]).
--export_type([outbox/0]).
-
--record(segment, {
-    id :: pos_integer(),
-    path :: file:filename_all(),
-    start_seq = 0 :: non_neg_integer(),
-    end_seq = 0 :: non_neg_integer(),
-    records = 0 :: non_neg_integer()
-}).
-
--record(outbox, {
-    dir :: file:filename_all(),
-    metadata_path :: file:filename_all(),
-    writer_segment = 1 :: pos_integer(),
-    fd :: file:fd(),
-    segment_record_limit :: pos_integer(),
-    max_segments :: pos_integer(),
-    max_record_bytes :: pos_integer(),
-    segments = [] :: [#segment{}],
-    next_seq = 1 :: pos_integer(),
-    write_seq = 0 :: non_neg_integer(),
-    acked_seq = 0 :: non_neg_integer()
-}).
-
--type outbox() :: #outbox{}.
-
--define(METADATA_FILE, "metadata.json").
--define(SEGMENT_PREFIX, "segment-").
--define(SEGMENT_EXT, ".log").
--define(RECORD_MAGIC, <<"EFKAOBX1">>).
--define(RECORD_MAGIC_SIZE, 8).
--define(RECORD_VERSION, 1).
--define(RECORD_HEADER_SIZE, 22).
--define(RECORD_CRC_SIZE, 4).
-
--type open_options() :: #{
-    dir := file:filename_all(),
-    segment_record_limit := pos_integer(),
-    max_segments := pos_integer(),
-    max_record_bytes := pos_integer()
-}.
-
--spec open(open_options()) -> {ok, outbox()} | {error, term()}.
-open(Options) when is_map(Options) ->
-    Dir = maps:get(dir, Options),
-    SegmentRecordLimit = positive_option(segment_record_limit, Options),
-    MaxSegments = positive_option(max_segments, Options),
-    MaxRecordBytes = positive_option(max_record_bytes, Options),
-    MetadataPath = filename:join(Dir, ?METADATA_FILE),
-    maybe
-        ok ?= ensure_dir(Dir),
-        {_MetaWriteSeq, MetaAckedSeq, MetaWriterSegment} = read_metadata(MetadataPath),
-        {ok, Segments} ?= load_segments(Dir, MaxRecordBytes),
-        ScannedWriteSeq = max_segment_end(Segments),
-        WriteSeq = ScannedWriteSeq,
-        AckedSeq = min(MetaAckedSeq, WriteSeq),
-        WriterSegment = writer_segment_id(Segments, MetaWriterSegment),
-        {ok, Fd} ?= open_writer(segment_path(Dir, WriterSegment)),
-        Outbox0 = #outbox{
-            dir = Dir,
-            metadata_path = MetadataPath,
-            writer_segment = WriterSegment,
-            fd = Fd,
-            segment_record_limit = SegmentRecordLimit,
-            max_segments = MaxSegments,
-            max_record_bytes = MaxRecordBytes,
-            segments = Segments,
-            next_seq = WriteSeq + 1,
-            write_seq = WriteSeq,
-            acked_seq = AckedSeq
-        },
-        {ok, Outbox1} ?= normalize_open_outbox(Outbox0),
-        ok ?= persist_metadata(Outbox1),
-        {ok, Outbox1}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec close(outbox()) -> ok.
-close(#outbox{fd = Fd}) ->
-    _ = file:close(Fd),
-    ok.
-
--spec append(binary(), outbox()) ->
-    {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}.
-append(Packet, Outbox0) when is_binary(Packet) ->
-    case ensure_writable_segment(Outbox0) of
-        {ok, Outbox1 = #outbox{fd = Fd, next_seq = Seq}} ->
-            Record = encode_record(Seq, Packet),
-            maybe
-                ok ?= file:write(Fd, Record),
-                ok ?= file:sync(Fd),
-                Outbox2 = mark_written(Outbox1, Seq),
-                ok ?= persist_metadata(Outbox2),
-                {ok, Outbox2}
-            else
-                {error, Reason} ->
-                    {error, Reason}
-            end;
-        {dropped, capacity_reached, Outbox1} ->
-            {dropped, capacity_reached, Outbox1};
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec next(outbox()) -> eof | {ok, pos_integer(), binary()} | {error, term()}.
-next(#outbox{segments = Segments, acked_seq = AckedSeq, max_record_bytes = MaxRecordBytes}) ->
-    case next_segment(Segments, AckedSeq) of
-        undefined ->
-            eof;
-        #segment{path = Path} ->
-            case file:open(Path, [read, binary]) of
-                {ok, Fd} ->
-                    Result = read_next_unacked(Fd, AckedSeq, MaxRecordBytes),
-                    _ = file:close(Fd),
-                    Result;
-                {error, enoent} ->
-                    eof;
-                {error, Reason} ->
-                    {error, Reason}
-            end
-    end.
-
--spec ack(pos_integer(), outbox()) -> {ok, outbox()} | {error, term()}.
-ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq =< AckedSeq ->
-    {ok, Outbox};
-ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq > AckedSeq ->
-    AckedOutbox = Outbox#outbox{acked_seq = Seq},
-    maybe
-        ok ?= persist_metadata(AckedOutbox),
-        {ok, NOutbox} ?= maybe_reset_or_prune(AckedOutbox),
-        ok ?= persist_metadata(NOutbox),
-        {ok, NOutbox}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end;
-ack(Seq, #outbox{acked_seq = AckedSeq}) when is_integer(Seq) ->
-    {error, {invalid_ack, Seq, AckedSeq}}.
-
--spec ensure_dir(file:filename_all()) -> ok | {error, term()}.
-ensure_dir(Dir) ->
-    filelib:ensure_dir(filename:join(Dir, "dummy")).
-
--spec positive_option(atom(), map()) -> pos_integer().
-positive_option(Key, Options) ->
-    case maps:get(Key, Options) of
-        Value when is_integer(Value), Value > 0 ->
-            Value;
-        Value ->
-            error({invalid_option, Key, Value})
-    end.
-
--spec read_metadata(file:filename_all()) ->
-    {non_neg_integer(), non_neg_integer(), pos_integer()}.
-read_metadata(MetadataPath) ->
-    case file:read_file(MetadataPath) of
-        {ok, Bin} ->
-            decode_metadata(Bin);
-        {error, _} ->
-            {0, 0, 1}
-    end.
-
--spec decode_metadata(binary()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}.
-decode_metadata(Bin) ->
-    try json:decode(Bin) of
-        Metadata ->
-            safe_metadata(Metadata)
-    catch
-        _:_Reason ->
-            {0, 0, 1}
-    end.
-
--spec safe_metadata(term()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}.
-safe_metadata(#{
-    <<"write_seq">> := WriteSeq,
-    <<"acked_seq">> := AckedSeq,
-    <<"writer_segment">> := WriterSegment
-})
-  when is_integer(WriteSeq), WriteSeq >= 0,
-       is_integer(AckedSeq), AckedSeq >= 0,
-       is_integer(WriterSegment), WriterSegment > 0 ->
-    {WriteSeq, AckedSeq, WriterSegment};
-safe_metadata(#{<<"write_seq">> := WriteSeq, <<"acked_seq">> := AckedSeq})
-  when is_integer(WriteSeq), WriteSeq >= 0, is_integer(AckedSeq), AckedSeq >= 0 ->
-    {WriteSeq, AckedSeq, 1};
-safe_metadata(_) ->
-    {0, 0, 1}.
-
--spec persist_metadata(outbox()) -> ok | {error, term()}.
-persist_metadata(#outbox{
-    metadata_path = MetadataPath,
-    writer_segment = WriterSegment,
-    write_seq = WriteSeq,
-    acked_seq = AckedSeq
-}) ->
-    Metadata = iolist_to_binary(json:encode(#{
-        <<"write_seq">> => WriteSeq,
-        <<"acked_seq">> => AckedSeq,
-        <<"writer_segment">> => WriterSegment
-    })),
-    TmpPath = MetadataPath ++ ".tmp",
-    case file:write_file(TmpPath, Metadata, [write, binary]) of
-        ok ->
-            file:rename(TmpPath, MetadataPath);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec load_segments(file:filename_all(), pos_integer()) -> {ok, [#segment{}]} | {error, term()}.
-load_segments(Dir, MaxRecordBytes) ->
-    case file:list_dir(Dir) of
-        {ok, Names} ->
-            Ids = lists:sort([Id || Name <- Names, {ok, Id} <- [parse_segment_id(Name)]]),
-            load_segments(Dir, Ids, MaxRecordBytes, []);
-        {error, enoent} ->
-            {ok, []};
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec load_segments(file:filename_all(), [pos_integer()], pos_integer(), [#segment{}]) ->
-    {ok, [#segment{}]} | {error, term()}.
-load_segments(_Dir, [], _MaxRecordBytes, Acc) ->
-    {ok, lists:reverse(Acc)};
-load_segments(Dir, [Id | Rest], MaxRecordBytes, Acc) ->
-    Path = segment_path(Dir, Id),
-    case scan_segment(Path, Id, MaxRecordBytes) of
-        {ok, undefined} ->
-            load_segments(Dir, Rest, MaxRecordBytes, Acc);
-        {ok, Segment} ->
-            load_segments(Dir, Rest, MaxRecordBytes, [Segment | Acc]);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec parse_segment_id(string()) -> {ok, pos_integer()} | error.
-parse_segment_id(Name) ->
-    case re:run(Name, "^" ++ ?SEGMENT_PREFIX ++ "([0-9]+)\\" ++ ?SEGMENT_EXT ++ "$",
-        [{capture, [1], list}])
-    of
-        {match, [Digits]} ->
-            case list_to_integer(Digits) of
-                Id when Id > 0 ->
-                    {ok, Id};
-                _ ->
-                    error
-            end;
-        nomatch ->
-            error
-    end.
-
--spec segment_path(file:filename_all(), pos_integer()) -> file:filename_all().
-segment_path(Dir, Id) ->
-    filename:join(Dir, ?SEGMENT_PREFIX ++ integer_to_list(Id) ++ ?SEGMENT_EXT).
-
--spec open_writer(file:filename_all()) -> {ok, file:fd()} | {error, term()}.
-open_writer(Path) ->
-    file:open(Path, [append, binary]).
-
--spec writer_segment_id([#segment{}], pos_integer()) -> pos_integer().
-writer_segment_id([], MetaWriterSegment) ->
-    max(1, MetaWriterSegment);
-writer_segment_id(Segments, _MetaWriterSegment) ->
-    (lists:last(Segments))#segment.id.
-
--spec max_segment_end([#segment{}]) -> non_neg_integer().
-max_segment_end([]) ->
-    0;
-max_segment_end(Segments) ->
-    lists:max([Segment#segment.end_seq || Segment <- Segments]).
-
--spec normalize_open_outbox(outbox()) -> {ok, outbox()} | {error, term()}.
-normalize_open_outbox(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) when Seq > 0 ->
-    reset_empty_outbox(Outbox);
-normalize_open_outbox(Outbox) ->
-    prune_acked_segments(Outbox).
-
--spec ensure_writable_segment(outbox()) ->
-    {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}.
-ensure_writable_segment(Outbox = #outbox{segments = Segments, writer_segment = WriterSegment}) ->
-    case find_segment(WriterSegment, Segments) of
-        undefined ->
-            {ok, Outbox};
-        #segment{records = Records} when Records < Outbox#outbox.segment_record_limit ->
-            {ok, Outbox};
-        #segment{} when length(Segments) >= Outbox#outbox.max_segments ->
-            {dropped, capacity_reached, Outbox};
-        #segment{} ->
-            rotate_writer(Outbox, WriterSegment + 1)
-    end.
-
--spec rotate_writer(outbox(), pos_integer()) -> {ok, outbox()} | {error, term()}.
-rotate_writer(Outbox = #outbox{dir = Dir, fd = Fd}, NewSegment) ->
-    maybe
-        ok ?= file:sync(Fd),
-        {ok, NFd} ?= open_writer(segment_path(Dir, NewSegment)),
-        _ = file:close(Fd),
-        {ok, Outbox#outbox{writer_segment = NewSegment, fd = NFd}}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec mark_written(outbox(), pos_integer()) -> outbox().
-mark_written(Outbox = #outbox{
-    dir = Dir,
-    writer_segment = WriterSegment,
-    segments = Segments
-}, Seq) ->
-    Path = segment_path(Dir, WriterSegment),
-    Segment0 = case find_segment(WriterSegment, Segments) of
-        undefined ->
-            #segment{id = WriterSegment, path = Path, start_seq = Seq};
-        Segment ->
-            Segment
-    end,
-    Records = Segment0#segment.records + 1,
-    Segment1 = Segment0#segment{end_seq = Seq, records = Records},
-    Outbox#outbox{
-        segments = replace_segment(Segment1, Segments),
-        next_seq = Seq + 1,
-        write_seq = Seq
-    }.
-
--spec replace_segment(#segment{}, [#segment{}]) -> [#segment{}].
-replace_segment(Segment, Segments) ->
-    lists:sort(
-        fun(A, B) -> A#segment.id < B#segment.id end,
-        [Segment | [S || S <- Segments, S#segment.id =/= Segment#segment.id]]
-    ).
-
--spec find_segment(pos_integer(), [#segment{}]) -> #segment{} | undefined.
-find_segment(Id, Segments) ->
-    case [Segment || Segment <- Segments, Segment#segment.id =:= Id] of
-        [Segment] ->
-            Segment;
-        [] ->
-            undefined
-    end.
-
--spec next_segment([#segment{}], non_neg_integer()) -> #segment{} | undefined.
-next_segment([], _AckedSeq) ->
-    undefined;
-next_segment([Segment = #segment{end_seq = EndSeq} | _Rest], AckedSeq) when EndSeq > AckedSeq ->
-    Segment;
-next_segment([_Segment | Rest], AckedSeq) ->
-    next_segment(Rest, AckedSeq).
-
--spec maybe_reset_or_prune(outbox()) -> {ok, outbox()} | {error, term()}.
-maybe_reset_or_prune(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) ->
-    reset_empty_outbox(Outbox);
-maybe_reset_or_prune(Outbox) ->
-    prune_acked_segments(Outbox).
-
--spec prune_acked_segments(outbox()) -> {ok, outbox()} | {error, term()}.
-prune_acked_segments(Outbox = #outbox{segments = Segments, acked_seq = AckedSeq}) ->
-    {DeleteSegments, KeepSegments} = take_acked_segments(Segments, AckedSeq, []),
-    maybe
-        ok ?= delete_files([Segment#segment.path || Segment <- DeleteSegments]),
-        {ok, Outbox#outbox{segments = KeepSegments}}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec take_acked_segments([#segment{}], non_neg_integer(), [#segment{}]) ->
-    {[#segment{}], [#segment{}]}.
-take_acked_segments([Segment = #segment{end_seq = EndSeq} | Rest], AckedSeq, Acc)
-  when EndSeq =< AckedSeq ->
-    take_acked_segments(Rest, AckedSeq, [Segment | Acc]);
-take_acked_segments(Segments, _AckedSeq, Acc) ->
-    {lists:reverse(Acc), Segments}.
-
--spec reset_empty_outbox(outbox()) -> {ok, outbox()} | {error, term()}.
-reset_empty_outbox(Outbox = #outbox{
-    dir = Dir,
-    fd = Fd,
-    writer_segment = WriterSegment,
-    segments = Segments
-}) ->
-    CurrentWriterPath = segment_path(Dir, WriterSegment),
-    SegmentPaths = [Segment#segment.path || Segment <- Segments],
-    Paths = lists:usort([CurrentWriterPath | SegmentPaths]),
-    maybe
-        ok ?= file:close(Fd),
-        ok ?= delete_files(Paths),
-        {ok, NFd} ?= open_writer(segment_path(Dir, 1)),
-        {ok, Outbox#outbox{
-            writer_segment = 1,
-            fd = NFd,
-            segments = [],
-            next_seq = 1,
-            write_seq = 0,
-            acked_seq = 0
-        }}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec delete_files([file:filename_all()]) -> ok | {error, term()}.
-delete_files([]) ->
-    ok;
-delete_files([Path | Rest]) ->
-    case file:delete(Path) of
-        ok ->
-            delete_files(Rest);
-        {error, enoent} ->
-            delete_files(Rest);
-        {error, Reason} ->
-            {error, {delete_failed, Path, Reason}}
-    end.
-
--spec scan_segment(file:filename_all(), pos_integer(), pos_integer()) ->
-    {ok, #segment{} | undefined} | {error, term()}.
-scan_segment(Path, Id, MaxRecordBytes) ->
-    case file:open(Path, [read, binary]) of
-        {ok, Fd} ->
-            Result = scan_segment(Fd, Id, Path, MaxRecordBytes, 0, 0, 0),
-            _ = file:close(Fd),
-            Result;
-        {error, enoent} ->
-            {ok, undefined};
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec scan_segment(file:fd(), pos_integer(), file:filename_all(), pos_integer(),
-    non_neg_integer(), non_neg_integer(), non_neg_integer()) ->
-    {ok, #segment{} | undefined} | {error, term()}.
-scan_segment(Fd, Id, Path, MaxRecordBytes, StartSeq, EndSeq, Records) ->
-    case read_record(Fd, MaxRecordBytes) of
-        eof when Records =:= 0 ->
-            {ok, undefined};
-        eof ->
-            {ok, #segment{
-                id = Id,
-                path = Path,
-                start_seq = StartSeq,
-                end_seq = EndSeq,
-                records = Records
-            }};
-        {ok, Seq, _Packet} ->
-            NStartSeq = case StartSeq of
-                0 -> Seq;
-                _ -> StartSeq
-            end,
-            scan_segment(Fd, Id, Path, MaxRecordBytes, NStartSeq, Seq, Records + 1);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec encode_record(pos_integer(), binary()) -> binary().
-encode_record(Seq, Packet) ->
-    PacketSize = byte_size(Packet),
-    Magic = ?RECORD_MAGIC,
-    Header = <<Magic/binary, ?RECORD_VERSION:8/unsigned, ?RECORD_HEADER_SIZE:8/unsigned,
-        PacketSize:32/unsigned-big, Seq:64/unsigned-big>>,
-    Crc32 = erlang:crc32(<<Header/binary, Packet/binary>>),
-    <<Header/binary, Packet/binary, Crc32:32/unsigned-big>>.
-
--spec read_next_unacked(file:fd(), non_neg_integer(), pos_integer()) ->
-    eof | {ok, pos_integer(), binary()} | {error, term()}.
-read_next_unacked(Fd, AckedSeq, MaxRecordBytes) ->
-    case read_record(Fd, MaxRecordBytes) of
-        eof ->
-            eof;
-        {ok, Seq, Packet} when Seq > AckedSeq ->
-            {ok, Seq, Packet};
-        {ok, _Seq, _Packet} ->
-            read_next_unacked(Fd, AckedSeq, MaxRecordBytes);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec read_record(file:fd(), pos_integer()) -> eof | {ok, pos_integer(), binary()} | {error, term()}.
-read_record(Fd, MaxRecordBytes) ->
-    case find_next_magic(Fd) of
-        eof ->
-            eof;
-        {ok, StartPos} ->
-            case read_record_after_magic(Fd, MaxRecordBytes) of
-                {ok, Seq, Packet} ->
-                    {ok, Seq, Packet};
-                {resync, _Reason} ->
-                    case file:position(Fd, {bof, StartPos + 1}) of
-                        {ok, _} ->
-                            read_record(Fd, MaxRecordBytes);
-                        {error, Reason} ->
-                            {error, Reason}
-                    end
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec find_next_magic(file:fd()) -> eof | {ok, non_neg_integer()} | {error, term()}.
-find_next_magic(Fd) ->
-    case file:position(Fd, cur) of
-        {ok, StartPos} ->
-            case file:read(Fd, ?RECORD_MAGIC_SIZE) of
-                eof ->
-                    eof;
-                {ok, Magic} when byte_size(Magic) < ?RECORD_MAGIC_SIZE ->
-                    eof;
-                {ok, ?RECORD_MAGIC} ->
-                    {ok, StartPos};
-                {ok, Window} ->
-                    find_next_magic(Fd, Window);
-                {error, Reason} ->
-                    {error, Reason}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec find_next_magic(file:fd(), binary()) -> eof | {ok, non_neg_integer()} | {error, term()}.
-find_next_magic(Fd, Window) ->
-    case file:read(Fd, 1) of
-        eof ->
-            eof;
-        {ok, Byte} ->
-            Tail = binary:part(Window, 1, ?RECORD_MAGIC_SIZE - 1),
-            NWindow = <<Tail/binary, Byte/binary>>,
-            case NWindow of
-                ?RECORD_MAGIC ->
-                    case file:position(Fd, cur) of
-                        {ok, Pos} ->
-                            {ok, Pos - ?RECORD_MAGIC_SIZE};
-                        {error, Reason} ->
-                            {error, Reason}
-                    end;
-                _ ->
-                    find_next_magic(Fd, NWindow)
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec read_record_after_magic(file:fd(), pos_integer()) ->
-    {ok, pos_integer(), binary()} | {resync, term()}.
-read_record_after_magic(Fd, MaxRecordBytes) ->
-    HeaderRestSize = ?RECORD_HEADER_SIZE - ?RECORD_MAGIC_SIZE,
-    case file:read(Fd, HeaderRestSize) of
-        {ok, <<Version:8/unsigned, HeaderSize:8/unsigned, PacketSize:32/unsigned-big,
-            Seq:64/unsigned-big>>} ->
-            read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, MaxRecordBytes);
-        {ok, Partial} ->
-            {resync, {truncated_record_header, Partial}};
-        eof ->
-            {resync, truncated_record_header};
-        {error, Reason} ->
-            {resync, Reason}
-    end.
-
--spec read_record_payload(file:fd(), non_neg_integer(), non_neg_integer(),
-    non_neg_integer(), non_neg_integer(), pos_integer()) ->
-    {ok, pos_integer(), binary()} | {resync, term()}.
-read_record_payload(_Fd, Version, _HeaderSize, _PacketSize, _Seq, _MaxRecordBytes)
-  when Version =/= ?RECORD_VERSION ->
-    {resync, {invalid_record_version, Version}};
-read_record_payload(_Fd, _Version, HeaderSize, _PacketSize, _Seq, _MaxRecordBytes)
-  when HeaderSize =/= ?RECORD_HEADER_SIZE ->
-    {resync, {invalid_record_header_size, HeaderSize}};
-read_record_payload(_Fd, _Version, _HeaderSize, PacketSize, _Seq, MaxRecordBytes)
-  when PacketSize =:= 0; PacketSize > MaxRecordBytes ->
-    {resync, {invalid_record_packet_size, PacketSize}};
-read_record_payload(_Fd, _Version, _HeaderSize, _PacketSize, Seq, _MaxRecordBytes)
-  when Seq =:= 0 ->
-    {resync, {invalid_record_seq, Seq}};
-read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, _MaxRecordBytes) ->
-    case file:read(Fd, PacketSize + ?RECORD_CRC_SIZE) of
-        {ok, <<Packet:PacketSize/binary, Crc32:32/unsigned-big>>} ->
-            validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32);
-        {ok, Partial} ->
-            {resync, {truncated_record_payload, PacketSize, byte_size(Partial)}};
-        eof ->
-            {resync, {truncated_record_payload, PacketSize, 0}};
-        {error, Reason} ->
-            {resync, Reason}
-    end.
-
--spec validate_record_crc(non_neg_integer(), non_neg_integer(), non_neg_integer(),
-    pos_integer(), binary(), non_neg_integer()) ->
-    {ok, pos_integer(), binary()} | {resync, term()}.
-validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32) ->
-    Magic = ?RECORD_MAGIC,
-    Header = <<Magic/binary, Version:8/unsigned, HeaderSize:8/unsigned,
-        PacketSize:32/unsigned-big, Seq:64/unsigned-big>>,
-    case erlang:crc32(<<Header/binary, Packet/binary>>) of
-        Crc32 ->
-            {ok, Seq, Packet};
-        Expected ->
-            {resync, {crc32_mismatch, Seq, Expected, Crc32}}
-    end.

apps/efka/src/iot/efka_iot_stream.erl

@@ -1,156 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc One transparent TCP stream from iot to the local manager service.
-%%% The iot/efka protocol does not inspect HTTP; all HTTP bytes are carried
-%%% in data frames.
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_stream).
--include("message.hrl").
-
--export([start_stream/2]).
--export([run/2]).
-
--define(DEFAULT_CONNECT_TIMEOUT, 3000).
--define(DEFAULT_IDLE_TIMEOUT, 120000).
-
--type stream_id() :: pos_integer().
--type stream_target() :: pos_integer().
-
--spec start_stream(StreamId :: integer(), Target :: stream_target()) -> {ok, {pid(), reference()}}.
-start_stream(StreamId, Target) when is_integer(StreamId), StreamId > 0, is_integer(Target), Target > 0 ->
-    {ok, spawn_monitor(?MODULE, run, [StreamId, Target])}.
-
--spec run(StreamId :: stream_id(), Target :: stream_target()) -> ok.
-run(StreamId, Target) when is_integer(StreamId), StreamId > 0, is_integer(Target), Target > 0 ->
-    try run0(StreamId, Target) of
-        ok ->
-            ok
-    catch
-        Class:Reason:Stack ->
-            logger:warning("[efka_iot_stream] stream_id: ~p crashed, class: ~p, reason: ~p, stack: ~p",
-                [StreamId, Class, Reason, Stack]),
-            efka_iot_client:send_stream(StreamId, {reset, safe_term({Class, Reason})}),
-            ok
-    after
-        efka_iot_client:stream_done(StreamId)
-    end.
-
--spec run0(stream_id(), stream_target()) -> ok.
-run0(StreamId, Target) ->
-    case open_target_socket(Target) of
-        {ok, Socket, IdleTimeout} ->
-            efka_iot_client:send_stream(StreamId, opened),
-            ok = inet:setopts(Socket, [{active, once}]),
-            loop(StreamId, Socket, IdleTimeout);
-        {error, Reason} ->
-            efka_iot_client:send_stream(StreamId, {open_error, safe_term(Reason)}),
-            ok
-    end.
-
--spec open_target_socket(stream_target()) -> {ok, gen_tcp:socket(), timeout()} | {error, term()}.
-open_target_socket(Target) ->
-    case stream_target_props(Target) of
-        {ok, Props} ->
-            open_target_socket(Target, Props);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec open_target_socket(stream_target(), proplists:proplist()) ->
-    {ok, gen_tcp:socket(), timeout()} | {error, term()}.
-open_target_socket(Target, Props) ->
-    Host = proplists:get_value(host, Props),
-    Port = proplists:get_value(port, Props),
-    ConnectTimeout = proplists:get_value(connect_timeout, Props, ?DEFAULT_CONNECT_TIMEOUT),
-    IdleTimeout = proplists:get_value(idle_timeout, Props, ?DEFAULT_IDLE_TIMEOUT),
-
-    SocketOpts = [
-        binary,
-        {packet, raw},
-        {active, false},
-        {nodelay, true}
-    ],
-    case gen_tcp:connect(Host, Port, SocketOpts, ConnectTimeout) of
-        {ok, Socket} ->
-            {ok, Socket, IdleTimeout};
-        {error, Reason} ->
-            {error, {connect_failed, Target, Reason}}
-    end.
-
--spec stream_target_props(stream_target()) -> {ok, proplists:proplist()} | {error, term()}.
-stream_target_props(Target) ->
-    case application:get_env(efka, stream_targets) of
-        {ok, Targets} ->
-            case proplists:get_value(Target, Targets) of
-                undefined ->
-                    {error, {unknown_stream_target, Target}};
-                Props ->
-                    {ok, Props}
-            end;
-        undefined when Target =:= ?STREAM_TARGET_MANAGER ->
-            case application:get_env(efka, stream_target) of
-                {ok, Props} ->
-                    {ok, Props};
-                undefined ->
-                    {error, {unknown_stream_target, Target}}
-            end;
-        undefined ->
-            {error, {unknown_stream_target, Target}}
-    end.
-
--spec loop(stream_id(), gen_tcp:socket(), timeout()) -> ok.
-loop(StreamId, Socket, IdleTimeout) ->
-    receive
-        {stream, StreamId, {data, Data}} when is_binary(Data) ->
-            case gen_tcp:send(Socket, Data) of
-                ok ->
-                    loop(StreamId, Socket, IdleTimeout);
-                {error, Reason} ->
-                    efka_iot_client:send_stream(StreamId, {reset, safe_term(Reason)}),
-                    close_socket(Socket)
-            end;
-        {stream, StreamId, fin} ->
-            _ = gen_tcp:shutdown(Socket, write),
-            loop(StreamId, Socket, IdleTimeout);
-        {stream, StreamId, {reset, _Reason}} ->
-            close_socket(Socket);
-        {tcp, Socket, Data} ->
-            efka_iot_client:send_stream(StreamId, {data, Data}),
-            ok = inet:setopts(Socket, [{active, once}]),
-            loop(StreamId, Socket, IdleTimeout);
-        {tcp_closed, Socket} ->
-            efka_iot_client:send_stream(StreamId, fin),
-            close_socket(Socket);
-        {tcp_error, Socket, Reason} ->
-            efka_iot_client:send_stream(StreamId, {reset, safe_term(Reason)}),
-            close_socket(Socket);
-        Info ->
-            logger:debug("[efka_iot_stream] stream_id: ~p ignore unknown info: ~p", [StreamId, Info]),
-            loop(StreamId, Socket, IdleTimeout)
-    after IdleTimeout ->
-        efka_iot_client:send_stream(StreamId, {reset, <<"idle_timeout">>}),
-        close_socket(Socket)
-    end.
-
--spec close_socket(gen_tcp:socket()) -> ok.
-close_socket(Socket) ->
-    catch gen_tcp:close(Socket),
-    ok.
-
--spec safe_term(term()) -> term().
-safe_term(true) ->
-    true;
-safe_term(false) ->
-    false;
-safe_term(undefined) ->
-    undefined;
-safe_term(Value) when is_atom(Value) ->
-    atom_to_binary(Value, utf8);
-safe_term(Value) when is_map(Value) ->
-    maps:from_list([{safe_term(K), safe_term(V)} || {K, V} <- maps:to_list(Value)]);
-safe_term(Value) when is_list(Value) ->
-    [safe_term(Item) || Item <- Value];
-safe_term(Value) when is_tuple(Value) ->
-    list_to_tuple([safe_term(Item) || Item <- tuple_to_list(Value)]);
-safe_term(Value) ->
-    Value.

apps/efka/src/models/cache_model.erl

@@ -4,35 +4,46 @@
 %%% @doc
 %%%
 %%% @end
-%%% Created : 16. 9月 2025 16:48
+%%% Created : 13. 8月 2025 16:05
 %%%-------------------------------------------------------------------
--module(docker_events).
+-module(cache_model).
 -author("anlicheng").
+-include("efka_tables.hrl").
 
 -behaviour(gen_server).
 
 %% API
 -export([start_link/0]).
--export([monitor_container/2]).
+-export([insert/2, fetch_next/0, delete/1, get_all_cache/0]).
 
 %% gen_server callbacks
 -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
 
 -define(SERVER, ?MODULE).
+-define(TAB, cache).
 
 -record(state, {
-    port,
+
-    %% 观察者
-    monitors = #{}
 }).
 
 %%%===================================================================
 %%% API
 %%%===================================================================
 
--spec monitor_container(ReceiverPid :: pid(), ContainerId :: binary()) -> ok.
+-spec insert(Method :: integer(), Data :: binary()) -> ok | {error, Reason :: any()}.
-monitor_container(ReceiverPid, ContainerId) when is_pid(ReceiverPid), is_binary(ContainerId) ->
+insert(Method, Data) when is_integer(Method), is_binary(Data) ->
-    gen_server:cast(?SERVER, {monitor_container, ReceiverPid, ContainerId}).
+    Cache = #cache{id = generate_id(), method = Method, data = Data},
+    gen_server:call(?SERVER, {insert, Cache}).
+
+fetch_next() ->
+    gen_server:call(?SERVER, fetch_next).
+
+delete(Id) when is_integer(Id) ->
+    gen_server:call(?SERVER, {delete, Id}).
+
+-spec get_all_cache() -> [#cache{}].
+get_all_cache() ->
+    gen_server:call(?SERVER, get_all_cache).
 
 %% @doc Spawns the server and registers the local name (unique)
 -spec(start_link() ->
@@ -50,8 +61,9 @@ start_link() ->
     {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term()} | ignore).
 init([]) ->
-    process_flag(trap_exit, true),
+    {ok, DetsDir} = application:get_env(efka, dets_dir),
-    try_attach_events(0),
+    File = DetsDir ++ "cache.dets",
+    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, bag}, {keypos, 2}]),
     {ok, #state{}}.
 
 %% @private
@@ -64,6 +76,25 @@ init([]) ->
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
     {stop, Reason :: term(), NewState :: #state{}}).
+handle_call({insert, Cache}, _From, State = #state{}) ->
+    ok = dets:insert(?TAB, Cache),
+    {reply, ok, State};
+handle_call(fetch_next, _From, State = #state{}) ->
+    case dets:first(?TAB) of
+        '$end_of_table' ->
+            {reply, error, State};
+        Key ->
+            [Entry] = dets:lookup(?TAB, Key),
+            {reply, {ok, Entry}, State}
+    end;
+handle_call({delete, Id}, _From, State = #state{}) ->
+    ok = dets:delete(?TAB, Id),
+    {reply, ok, State};
+
+handle_call(get_all_cache, _From, State = #state{}) ->
+    Items = dets:foldl(fun(Record, Acc) -> [Record|Acc] end, [], ?TAB),
+    {reply, {ok, lists:reverse(Items)}, State};
+
 handle_call(_Request, _From, State = #state{}) ->
     {reply, ok, State}.
 
@@ -73,9 +104,8 @@ handle_call(_Request, _From, State = #state{}) ->
     {noreply, NewState :: #state{}} |
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast({monitor_container, ReceiverPid, ContainerId}, State = #state{monitors = Monitors}) ->
+handle_cast(_Request, State = #state{}) ->
-    MRef = erlang:monitor(process, ReceiverPid),
+    {noreply, State}.
-    {noreply, State#state{monitors = maps:put(ContainerId, {ReceiverPid, MRef}, Monitors)}}.
 
 %% @private
 %% @doc Handling all non call/cast messages
@@ -83,31 +113,8 @@ handle_cast({monitor_container, ReceiverPid, ContainerId}, State = #state{monito
     {noreply, NewState :: #state{}} |
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_info({timeout, _, attach_docker_events}, State = #state{port = undefined}) ->
+handle_info(_Info, State = #state{}) ->
-    ExecCmd = "docker events --format \"{{json .}}\"",
+    {noreply, State}.
-    case catch erlang:open_port({spawn, ExecCmd}, [exit_status, {line, 10239}, use_stdio, stderr_to_stdout, binary]) of
-        Port when is_port(Port) ->
-            {noreply, State#state{port = Port}};
-        _Error ->
-            try_attach_events(5000),
-            {noreply, State}
-    end;
-handle_info({Port, {data, {eol, BinLine}}}, State = #state{port = Port}) ->
-    Event = catch json:decode(BinLine),
-    logger:debug("event: ~p", [Event]),
-    handle_event(Event, State),
-    {noreply, State};
-
-%% 进程退出的时候删除掉管理的Pid
-handle_info({'DOWN', MRef, process, _Pid, _Reason}, State = #state{monitors = Monitors}) ->
-    NMonitors = maps:filter(fun(_Key, {_, Ref}) -> MRef =/= Ref end, Monitors),
-    {noreply, State#state{monitors = NMonitors}};
-
-%% Port退出的时候，尝试重启
-handle_info({'EXIT', Port, Reason}, State = #state{port = Port}) ->
-    logger:warning("[efka_docker_events] exit with reason: ~p", [Reason]),
-    try_attach_events(5000),
-    {noreply, State#state{port = undefined}}.
 
 %% @private
 %% @doc This function is called by a gen_server when it is about to
@@ -117,6 +124,7 @@ handle_info({'EXIT', Port, Reason}, State = #state{port = Port}) ->
 -spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
     State :: #state{}) -> term()).
 terminate(_Reason, _State = #state{}) ->
+    dets:close(?TAB),
     ok.
 
 %% @private
@@ -130,24 +138,7 @@ code_change(_OldVsn, State = #state{}, _Extra) ->
 %%%===================================================================
 %%% Internal functions
 %%%===================================================================
--spec handle_event(term(), #state{}) -> ok.
-handle_event(#{<<"Type">> := <<"container">>, <<"status">> := Status, <<"id">> := Id}, #state{monitors = Monitors}) ->
-    case maps:find(Id, Monitors) of
-        error ->
-            ok;
-        {ok, {ReceiverPid, _}} ->
-            case Status of
-                <<"start">> ->
-                    ReceiverPid ! {docker_events, start};
-                <<"stop">> ->
-                    ReceiverPid ! {docker_events, stop};
-                _ ->
-                    ok
-            end
-    end;
-handle_event(_, _) ->
-    ok.
 
--spec try_attach_events(non_neg_integer()) -> reference().
+-spec generate_id() -> integer().
-try_attach_events(Timeout) ->
+generate_id() ->
-    erlang:start_timer(Timeout, self(), attach_docker_events).
+    os:system_time(microsecond).

apps/efka/src/models/efka_model_sup.erl

@@ -0,0 +1,74 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 18. 4月 2025 16:42
+%%%-------------------------------------------------------------------
+-module(efka_model_sup).
+-author("anlicheng").
+-include("efka_tables.hrl").
+
+-behaviour(supervisor).
+
+%% API
+-export([start_link/0]).
+
+%% Supervisor callbacks
+-export([init/1]).
+
+-define(SERVER, ?MODULE).
+
+%%%===================================================================
+%%% API functions
+%%%===================================================================
+
+%% @doc Starts the supervisor
+-spec(start_link() -> {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link() ->
+    supervisor:start_link({local, ?SERVER}, ?MODULE, []).
+
+%%%===================================================================
+%%% Supervisor callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Whenever a supervisor is started using supervisor:start_link/[2,3],
+%% this function is called by the new process to find out about
+%% restart strategy, maximum restart frequency and child
+%% specifications.
+init([]) ->
+    SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
+    Specs = [
+        #{
+            id => cache_model,
+            start => {cache_model, start_link, []},
+            restart => permanent,
+            shutdown => 5000,
+            type => worker,
+            modules => ['cache_model']
+        },
+        #{
+            id => service_model,
+            start => {service_model, start_link, []},
+            restart => permanent,
+            shutdown => 5000,
+            type => worker,
+            modules => ['service_model']
+        },
+        #{
+            id => task_log_model,
+            start => {task_log_model, start_link, []},
+            restart => permanent,
+            shutdown => 5000,
+            type => worker,
+            modules => ['task_log_model']
+        }
+    ],
+
+    {ok, {SupFlags, Specs}}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================

apps/efka/src/models/service_model.erl

@@ -6,7 +6,7 @@
 %%% @end
 %%% Created : 13. 8月 2025 16:41
 %%%-------------------------------------------------------------------
--module(efka_service_model).
+-module(service_model).
 -author("anlicheng").
 -include("efka_tables.hrl").
 
@@ -14,7 +14,7 @@
 
 %% API
 -export([start_link/0]).
--export([insert/1, change_status/2, get_status/1, get_service/1, get_all_services/0, get_running_services/0]).
+-export([insert/1, change_status/2, set_config/2, get_config_json/1, get_status/1, get_service/1, get_all_services/0, get_running_services/0]).
 
 %% gen_server callbacks
 -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
@@ -30,14 +30,20 @@
 %%% API
 %%%===================================================================
 
--spec insert(#service{}) -> ok.
 insert(Service = #service{}) ->
     gen_server:call(?SERVER, {insert, Service}).
 
--spec change_status(binary(), integer()) -> ok | {error, binary()}.
 change_status(ServiceId, NewStatus) when is_binary(ServiceId), is_integer(NewStatus) ->
     gen_server:call(?SERVER, {change_status, ServiceId, NewStatus}).
 
+-spec set_config(ServiceId :: binary(), ConfigJson :: binary()) -> ok | {error, Reason :: any()}.
+set_config(ServiceId, ConfigJson) when is_binary(ServiceId), is_binary(ConfigJson) ->
+    gen_server:call(?SERVER, {set_config, ServiceId, ConfigJson}).
+
+-spec get_config_json(ServiceId :: binary()) -> error | {ok, ConfigJson :: binary()}.
+get_config_json(ServiceId) when is_binary(ServiceId) ->
+    gen_server:call(?SERVER, {get_config_json, ServiceId}).
+
 -spec get_status(ServiceId :: binary()) -> Status :: integer().
 get_status(ServiceId) when is_binary(ServiceId) ->
     gen_server:call(?SERVER, {get_status, ServiceId}).
@@ -72,7 +78,7 @@ start_link() ->
 init([]) ->
     {ok, DetsDir} = application:get_env(efka, dets_dir),
     File = DetsDir ++ "service.dets",
-    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, set}, {keypos, 2}]),
+    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, bag}, {keypos, 2}]),
     {ok, #state{}}.
 
 %% @private
@@ -85,18 +91,8 @@ init([]) ->
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
     {stop, Reason :: term(), NewState :: #state{}}).
-handle_call({insert, Service = #service{service_id = ServiceId}}, _From, State = #state{}) ->
+handle_call({insert, Service}, _From, State = #state{}) ->
-    case dets:lookup(?TAB, ServiceId) of
+    ok = dets:insert(?TAB, Service),
-        [] ->
-            ok = dets:insert(?TAB, Service);
-        [OldService] ->
-            NewService = OldService#service{
-                meta_data = Service#service.meta_data,
-                container_name = Service#service.container_name,
-                update_ts = Service#service.update_ts
-            },
-            ok = dets:insert(?TAB, NewService)
-    end,
     {reply, ok, State};
 
 handle_call({change_status, ServiceId, NewStatus}, _From, State = #state{}) ->
@@ -109,6 +105,24 @@ handle_call({change_status, ServiceId, NewStatus}, _From, State = #state{}) ->
             {reply, ok, State}
     end;
 
+handle_call({set_config, ServiceId, ConfigJson}, _From, State = #state{}) ->
+    case dets:lookup(?TAB, ServiceId) of
+        [] ->
+            {reply, {error, <<"service not found">>}, State};
+        [OldService] ->
+            NewService = OldService#service{config_json = ConfigJson},
+            ok = dets:insert(?TAB, NewService),
+            {reply, ok, State}
+    end;
+
+handle_call({get_config_json, ServiceId}, _From, State = #state{}) ->
+    case dets:lookup(?TAB, ServiceId) of
+        [] ->
+            {reply, error, State};
+        [#service{config_json = ConfigJson}] ->
+            {reply, {ok, ConfigJson}, State}
+    end;
+
 handle_call({get_status, ServiceId}, _From, State = #state{}) ->
     case dets:lookup(?TAB, ServiceId) of
         [] ->

apps/efka/src/models/task_log_model.erl

@@ -0,0 +1,122 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 13. 8月 2025 17:01
+%%%-------------------------------------------------------------------
+-module(task_log_model).
+-author("anlicheng").
+-include("efka_tables.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0]).
+-export([insert/2, get_logs/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+-define(TAB, task_log).
+
+-record(state, {
+
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+-spec insert(TaskId :: integer(), Logs :: [binary()]) -> ok | {error, Reason :: term()}.
+insert(TaskId, Logs) when is_integer(TaskId), is_list(Logs) ->
+    TaskLog = #task_log{task_id = TaskId, logs = Logs},
+    gen_server:call(?SERVER, {insert, TaskLog}).
+
+-spec get_logs(TaskId :: integer()) -> Logs :: [binary()].
+get_logs(TaskId) when is_integer(TaskId) ->
+    gen_server:call(?SERVER, {get_logs, TaskId}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link() ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([]) ->
+    {ok, DetsDir} = application:get_env(efka, dets_dir),
+    File = DetsDir ++ "task_log.dets",
+    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, set}, {keypos, 2}]),
+    {ok, #state{}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call({insert, TaskLog}, _From, State = #state{}) ->
+    ok = dets:insert(?TAB, TaskLog),
+    {reply, ok, State};
+handle_call({get_logs, TaskId}, _From, State = #state{}) ->
+    Reply = case dets:lookup(?TAB, TaskId) of
+                [] ->
+                    [];
+                [#task_log{logs = Logs}|_] ->
+                    Logs
+            end,
+    {reply, Reply, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================

apps/efka/src/service/efka_service.erl

@@ -1,145 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%% 1. 需要管理服务的整个生命周期，包括: 启动，停止
-%%% 2. 需要监控服务的状态，通过port的方式
-%%% 3. 服务的启动和关闭，需要在更高的层级控制
-%%% @end
-%%% Created : 18. 4月 2025 16:50
-%%%-------------------------------------------------------------------
--module(efka_service).
--author("anlicheng").
--include("efka_tables.hrl").
-
--behaviour(gen_server).
-
-%% API
--export([start_link/2]).
--export([get_name/1, get_pid/1, attach_channel/2]).
--export([metric_data/3, send_event/3]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--record(state, {
-    service_id :: binary(),
-    %% 通道id信息
-    channel_pid :: pid() | undefined
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec get_name(ServiceId :: binary()) -> atom().
-get_name(ServiceId) when is_binary(ServiceId) ->
-    list_to_atom("efka_service:" ++ binary_to_list(ServiceId)).
-
--spec get_pid(ServiceId :: binary()) -> undefined | pid().
-get_pid(ServiceId) when is_binary(ServiceId) ->
-    whereis(get_name(ServiceId)).
-
--spec metric_data(Pid :: pid(), RouteKey :: binary(), Metric :: binary()) -> ok.
-metric_data(Pid, RouteKey, Metric) when is_pid(Pid), is_binary(RouteKey),  is_binary(Metric) ->
-    gen_server:cast(Pid, {metric_data, RouteKey, Metric}).
-
--spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> ok.
-send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) ->
-    gen_server:cast(Pid, {send_event, EventType, Params}).
-
--spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}.
-attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) ->
-    gen_server:call(Pid, {attach_channel, ChannelPid}).
-
-%% @doc Spawns the server and registers the local name (unique)
--spec(start_link(Name :: atom(), Service :: binary()) ->
-    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
-start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) ->
-    gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
-%% @private
-%% @doc Initializes the server
--spec(init(Args :: term()) ->
-    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term()} | ignore).
-init([ServiceId]) ->
-    %% supervisor进程通过exit(ChildPid, shutdown)调用的时候，确保terminate函数被调用
-    logger:debug("[efka_service] service_id: ~p, started", [ServiceId]),
-    {ok, #state{service_id = ServiceId}}.
-
-%% @private
-%% @doc Handling call messages
--spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
-    State :: #state{}) ->
-    {reply, Reply :: term(), NewState :: #state{}} |
-    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 绑定channel
-handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) ->
-    case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of
-        false ->
-            erlang:monitor(process, ChannelPid),
-            logger:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
-            {reply, ok, State#state{channel_pid = ChannelPid}};
-        true ->
-            {reply, {error, <<"channel exists">>}, State}
-    end;
-
-handle_call(_Request, _From, State = #state{}) ->
-    {reply, ok, State}.
-
-%% @private
-%% @doc Handling cast messages
--spec(handle_cast(Request :: term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-handle_cast({metric_data, RouteKey, Metric}, State = #state{service_id = ServiceId}) ->
-    logger:debug("[efka_service] metric_data service_id: ~p, route_key: ~p, metric data: ~p", [ServiceId, RouteKey, Metric]),
-    efka_iot_client:metric_data(RouteKey, Metric),
-    {noreply, State};
-
-handle_cast(_Request, State = #state{}) ->
-    {noreply, State}.
-
-%% @private
-%% @doc Handling all non call/cast messages
--spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
-    {noreply, NewState :: #state{}} |
-    {noreply, NewState :: #state{}, timeout() | hibernate} |
-    {stop, Reason :: term(), NewState :: #state{}}).
-%% 处理channel进程的退出
-handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) ->
-    logger:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
-    {noreply, State#state{channel_pid = undefined}}.
-
-%% @private
-%% @doc This function is called by a gen_server when it is about to
-%% terminate. It should be the opposite of Module:init/1 and do any
-%% necessary cleaning up. When it returns, the gen_server terminates
-%% with Reason. The return value is ignored.
--spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
-    State :: #state{}) -> term()).
-terminate(Reason, _State = #state{service_id = ServiceId}) ->
-    logger:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
-    ok.
-
-%% @private
-%% @doc Convert process state when code is changed
--spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
-    Extra :: term()) ->
-    {ok, NewState :: #state{}} | {error, Reason :: term()}).
-code_change(_OldVsn, State = #state{}, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================

apps/efka/src/service/efka_service_channel.erl

@@ -1,169 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author licheng5
-%%% @copyright (C) 2021, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 11. 1月 2021 上午12:17
-%%%-------------------------------------------------------------------
--module(efka_service_channel).
--author("licheng5").
--include("efka_tables.hrl").
--include("service_pb.hrl").
-
-%% 一级帧类型
-%% REQUEST: 需要响应的请求帧
-%% RESPONSE: 对 REQUEST 的响应帧
-%% CAST: 单向消息帧，不需要响应
--define(FRAME_REQUEST, 16#01).
--define(FRAME_REPLY, 16#02).
--define(FRAME_CAST, 16#03).
-
-%% API
--export([init/2]).
--export([websocket_init/1, websocket_handle/2, websocket_info/2, terminate/3]).
-
--record(state, {
-    service_id :: undefined | binary(),
-    service_pid :: undefined | pid(),
-    subscribed_topics = sets:new(),
-    is_registered = false :: boolean()
-}).
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%% 逻辑处理方法
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
--spec init(term(), term()) -> {cowboy_websocket, term(), term()}.
-init(Req, Opts) ->
-    {cowboy_websocket, Req, Opts}.
-
--spec websocket_init(term()) -> {ok, #state{}}.
-websocket_init(_State) ->
-    logger:debug("[efka_service_channel] get a new connection"),
-    %% 初始状态为true
-    {ok, #state{}}.
-
--spec websocket_handle(term(), #state{}) ->
-    {reply, term(), #state{}} | {ok, #state{}}.
-websocket_handle(ping, State) ->
-    {reply, pong, State};
-
-websocket_handle({binary, <<?FRAME_REQUEST, PacketBin/binary>>}, State) ->
-    Request = service_pb:decode_msg(PacketBin, 'ServiceRequest'),
-    logger:debug("[efka_service_channel] get request: ~p", [Request]),
-    handle_request(Request, State);
-websocket_handle({binary, <<?FRAME_CAST, PacketBin/binary>>}, State) ->
-    Cast = service_pb:decode_msg(PacketBin, 'ServiceCast'),
-    logger:debug("[efka_service_channel] get cast: ~p", [Cast]),
-    handle_cast(Cast, State);
-
-websocket_handle(Info, State) ->
-    logger:error("[efka_service_channel] get a unknown message: ~p, channel will closed", [Info]),
-    {ok, State}.
-
-%% 订阅的消息
--spec websocket_info(term(), #state{}) ->
-    {reply, term(), #state{}} | {stop, #state{}} | {ok, #state{}}.
-websocket_info({topic_broadcast, Topic, Content}, State = #state{}) ->
-    Packet = service_pb:encode_msg(#'ServiceCast'{
-        body = {topic_event, #'ServiceCast.TopicEvent'{topic = Topic, content = Content}}
-    }),
-    logger:debug("[efka_service_channel] will publish topic: ~p", [Topic]),
-    {reply, {binary, <<?FRAME_CAST, Packet/binary>>}, State};
-
-%% service进程关闭
-websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) ->
-    logger:debug("[efka_service_channel] container_pid: ~p, exited: ~p", [ServicePid, Reason]),
-    {stop, State#state{service_pid = undefined}};
-
-%% 处理关闭信号
-websocket_info({stop, Reason}, State) ->
-    logger:debug("[efka_service_channel] the channel will be closed with reason: ~p", [Reason]),
-    {stop, State};
-
-%% 处理其他未知消息
-websocket_info(Info, State) ->
-    logger:debug("[efka_service_channel] channel get unknown info: ~p", [Info]),
-    {ok, State}.
-
-%% 进程关闭事件
--spec terminate(term(), term(), #state{}) -> ok.
-terminate(Reason, _Req, State = #state{service_id = ServiceId, is_registered = IsRegistered}) ->
-    ok = efka_subscription:unsubscribe_all(self()),
-    case IsRegistered of
-        true ->
-            ok = efka_service_model:change_status(ServiceId, 0);
-        false ->
-            ok
-    end,
-    logger:debug("[efka_service_channel] channel close with reason: ~p, state is: ~p", [Reason, State]),
-    ok.
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%% helper methods
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-%% 注册, 要建立程序和容器之间的关系
--spec handle_request(service_pb:'ServiceRequest'(), #state{}) -> {reply, {binary, binary()}, #state{}}.
-handle_request(#'ServiceRequest'{packet_id = PacketId, request = {register, #'ServiceRequest.Register'{service_id = ServiceId}}}, State) ->
-    {ok, ServicePid} = efka_service_sup:start_service(ServiceId),
-    case efka_service:attach_channel(ServicePid, self()) of
-        ok ->
-            erlang:monitor(process, ServicePid),
-
-            %% 更新微服务的状态
-            ok = efka_service_model:insert(#service{
-                service_id = ServiceId,
-                container_name = <<>>,
-                status = ?SERVICE_RUNNING,
-                meta_data = #{},
-                create_ts = efka_util:timestamp(),
-                update_ts = efka_util:timestamp()
-            }),
-
-            {reply, {binary, result_reply_packet(PacketId, <<"ok">>)},
-                State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}};
-        {error, Error} ->
-            logger:warning("[efka_service_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]),
-            {reply, {binary, error_reply_packet(PacketId, -1, <<"attach channel failed">>)}, State}
-    end;
-
-%% 订阅事件
-handle_request(#'ServiceRequest'{packet_id = PacketId, request = {subscribe, #'ServiceRequest.Subscribe'{topic = Topic}}},
-    State = #state{subscribed_topics = SubscribedTopics, is_registered = true}) ->
-
-    case efka_subscription:subscribe(Topic, self()) of
-        ok ->
-            Packet = result_reply_packet(PacketId, <<"ok">>),
-            {reply, {binary, Packet}, State#state{subscribed_topics = sets:add_element(Topic, SubscribedTopics)}};
-        {error, Reason} ->
-            Packet = error_reply_packet(PacketId, -1, Reason),
-            {reply, {binary, Packet}, State}
-    end;
-handle_request(#'ServiceRequest'{packet_id = PacketId}, State) ->
-    {reply, {binary, error_reply_packet(PacketId, -1, <<"invalid request">>)}, State}.
-
--spec handle_cast(service_pb:'ServiceCast'(), #state{}) -> {ok, #state{}}.
-handle_cast(#'ServiceCast'{body = {metric_data, #'ServiceCast.MetricData'{route_key = RouteKey, metric = Metric}}},
-    State = #state{service_pid = ServicePid, is_registered = true}) ->
-    efka_service:metric_data(ServicePid, RouteKey, Metric),
-    {ok, State};
-handle_cast(#'ServiceCast'{body = _Body}, State) ->
-    {ok, State}.
-
--spec result_reply_packet(integer(), binary()) -> binary().
-result_reply_packet(PacketId, Result) when is_integer(PacketId), is_binary(Result) ->
-    Reply = service_pb:encode_msg(#'ServiceReply'{
-        packet_id = PacketId,
-        reply = {result, Result}
-    }),
-    <<?FRAME_REPLY, Reply/binary>>.
-
--spec error_reply_packet(integer(), integer(), binary()) -> binary().
-error_reply_packet(PacketId, Code, Message) when is_integer(PacketId), is_integer(Code), is_binary(Message) ->
-    Reply = service_pb:encode_msg(#'ServiceReply'{
-        packet_id = PacketId,
-        reply = {error, #'ServiceReply.Error'{code = Code, message = Message}}
-    }),
-    <<?FRAME_REPLY, Reply/binary>>.

apps/efka/src/tcp_server/tcp_channel.erl

@@ -0,0 +1,201 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 30. 4月 2025 09:22
+%%%-------------------------------------------------------------------
+-module(tcp_channel).
+-author("anlicheng").
+-include("efka_service.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+%% 最大的等待时间
+-define(PENDING_TIMEOUT, 10 * 1000).
+%% 消息类型
+
+-record(state, {
+    packet_id = 1,
+    socket :: gen_tcp:socket(),
+    service_id :: undefined | binary(),
+    service_pid :: undefined | pid(),
+    is_registered = false :: boolean(),
+
+    %% 请求响应的对应关系, #{packet_id => {ReceiverPid, Ref}}; 自身的inflight需要超时逻辑处理
+    inflight = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link(Socket :: gen_tcp:socket()) ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link(Socket) ->
+    gen_server:start_link(?MODULE, [Socket], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([Socket]) ->
+    ok = inet:setopts(Socket, [{active, true}]),
+    lager:debug("[efka_tcp_channel] get micro service socket: ~p", [Socket]),
+    {ok, #state{socket = Socket}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+
+%% 推送配置项目
+handle_info({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) ->
+    PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"push_config">>, <<"params">> => #{<<"config">> => ConfigJson}},
+    Packet = jiffy:encode(PushConfig, [force_utf8]),
+    ok = gen_tcp:send(Socket, <<?PACKET_PUSH:8, Packet/binary>>),
+
+    erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}),
+    {noreply, State#state{packet_id = gen_channel:next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}};
+
+%% 远程调用
+handle_info({invoke, Ref, ReceiverPid, Payload}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) ->
+    PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"invoke">>, <<"params">> => #{<<"payload">> => Payload}},
+    Packet = jiffy:encode(PushConfig, [force_utf8]),
+    ok = gen_tcp:send(Socket, <<?PACKET_PUSH:8, Packet/binary>>),
+
+    erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}),
+    {noreply, State#state{packet_id = gen_channel:next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}};
+
+%% 处理micro-client:request => efka 主动的请求
+handle_info({tcp, Socket, <<?PACKET_REQUEST:8, Data/binary>>}, State = #state{socket = Socket}) ->
+    Request = jiffy:decode(Data, [return_maps]),
+    handle_request(Request, State);
+
+%% 处理micro-client:response => efka 的响应
+handle_info({tcp, Socket, <<?PACKET_RESPONSE:8, Data/binary>>}, State = #state{socket = Socket, inflight = Inflight}) ->
+    Resp = jiffy:decode(Data, [return_maps]),
+    {PacketId, Reply} = case Resp of
+                            #{<<"id">> := Id, <<"result">> := Result} ->
+                                {Id, {ok, Result}};
+                            #{<<"id">> := Id, <<"error">> := #{<<"code">> := _Code, <<"message">> := Error}} ->
+                                {Id, {error, Error}}
+                        end,
+
+    NInflight = gen_channel:channel_reply(PacketId, Reply, Inflight),
+    {noreply, State#state{inflight = NInflight}};
+
+%% 超时逻辑处理
+handle_info({timeout, _, {pending_timeout, Id}}, State = #state{inflight = Inflight}) ->
+    NInflight = gen_channel:channel_reply(Id, {error, <<"timeout">>}, Inflight),
+    {noreply, State#state{inflight = NInflight}};
+
+%% 订阅的消息
+handle_info({topic_broadcast, Topic, Content}, State = #state{socket = Socket}) ->
+    Packet = jiffy:encode(#{<<"topic">> => Topic, <<"content">> => Content}, [force_utf8]),
+    ok = gen_tcp:send(Socket, <<?PACKET_PUB:8, Packet/binary>>),
+
+    {noreply, State};
+
+%% service进程关闭
+handle_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) ->
+    lager:debug("[tcp_channel] service_pid: ~p, exited: ~p", [ServicePid, Reason]),
+    {stop, normal, State#state{service_pid = undefined}};
+
+handle_info({tcp_error, Socket, Reason}, State = #state{socket = Socket, service_id = ServiceId}) ->
+    lager:debug("[tcp_channel] tcp_error: ~p, assoc service: ~p", [Reason, ServiceId]),
+    {stop, normal, State};
+handle_info({tcp_closed, Socket}, State = #state{socket = Socket, service_id = ServiceId}) ->
+    lager:debug("[tcp_channel] tcp_closed: ~p, assoc service: ~p", [Socket, ServiceId]),
+    {stop, normal, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+%% 注册
+handle_request(#{<<"id">> := Id, <<"method">> := <<"register">>, <<"params">> := #{<<"service_id">> := ServiceId}}, State = #state{socket = Socket}) ->
+    case gen_channel:register(Id, ServiceId) of
+        {error, Reply} ->
+            ok = gen_tcp:send(Socket, <<?PACKET_RESPONSE:8, Reply/binary>>),
+            {stop, normal, State};
+        {ok, Reply, ServicePid} ->
+            ok = gen_tcp:send(Socket, <<?PACKET_RESPONSE:8, Reply/binary>>),
+            {noreply, State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}}
+    end;
+%% 请求参数
+handle_request(#{<<"id">> := Id, <<"method">> := <<"request_config">>}, State = #state{socket = Socket, service_pid = ServicePid, is_registered = true}) ->
+    {ok, ConfigJson} = efka_service:request_config(ServicePid),
+    Packet = gen_channel:json_result(Id, ConfigJson),
+    ok = gen_tcp:send(Socket, <<?PACKET_RESPONSE:8, Packet/binary>>),
+    {noreply, State};
+%% 数据项
+handle_request(#{<<"id">> := 0, <<"method">> := <<"metric_data">>,
+    <<"params">> := #{<<"device_uuid">> := DeviceUUID, <<"route_key">> := RouteKey, <<"metric">> := Metric}}, State = #state{service_pid = ServicePid, is_registered = true}) ->
+    efka_service:metric_data(ServicePid, DeviceUUID, RouteKey, Metric),
+    {noreply, State};
+%% Event事件
+handle_request(#{<<"id">> := 0, <<"method">> := <<"event">>, <<"params">> := #{<<"event_type">> := EventType, <<"body">> := Body}}, State = #state{service_pid = ServicePid, is_registered = true}) ->
+    efka_service:send_event(ServicePid, EventType, Body),
+    {noreply, State};
+%% 订阅事件
+handle_request(#{<<"id">> := 0, <<"method">> := <<"subscribe">>, <<"params">> := #{<<"topic">> := Topic}}, State = #state{is_registered = true}) ->
+    efka_subscription:subscribe(Topic, self()),
+    {noreply, State}.

apps/efka/src/tcp_server/tcp_connection_sup.erl

@@ -0,0 +1,43 @@
+%%%-------------------------------------------------------------------
+%% @doc efka top level supervisor.
+%% @end
+%%%-------------------------------------------------------------------
+
+-module(tcp_connection_sup).
+
+-behaviour(supervisor).
+
+-export([start_link/0, start_child/1]).
+
+-export([init/1]).
+
+-define(SERVER, ?MODULE).
+
+start_link() ->
+    supervisor:start_link({local, ?SERVER}, ?MODULE, []).
+
+%% sup_flags() = #{strategy => strategy(),         % optional
+%%                 intensity => non_neg_integer(), % optional
+%%                 period => pos_integer()}        % optional
+%% child_spec() = #{id => child_id(),       % mandatory
+%%                  start => mfargs(),      % mandatory
+%%                  restart => restart(),   % optional
+%%                  shutdown => shutdown(), % optional
+%%                  type => worker(),       % optional
+%%                  modules => modules()}   % optional
+init([]) ->
+    SupFlags = #{strategy => simple_one_for_one, intensity => 0, period => 1},
+    ChildSpec = #{
+        id => tcp_channel,
+        start => {tcp_channel, start_link, []},
+        restart => temporary,
+        type => worker
+    },
+    {ok, {SupFlags, [ChildSpec]}}.
+
+%% internal functions
+
+start_child(Socket) ->
+    supervisor:start_child(?MODULE, [Socket]).
+
+

apps/efka/src/tcp_server/tcp_server.erl

@@ -0,0 +1,46 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 29. 4月 2025 23:24
+%%%-------------------------------------------------------------------
+-module(tcp_server).
+-author("anlicheng").
+
+%% API
+-export([start_link/0, init/0]).
+
+start_link() ->
+    {ok, spawn_link(?MODULE, init, [])}.
+
+%% 监听循环
+init() ->
+    {ok, TcpServerProps} = application:get_env(efka, tcp_server),
+    Port = proplists:get_value(port, TcpServerProps),
+
+    case gen_tcp:listen(Port, [binary, {packet, 4}, {active, false}, {reuseaddr, true}]) of
+        {ok, ListenSocket} ->
+            lager:debug("[efka_tcp_server] Server started on port ~p~n", [Port]),
+            main_loop(ListenSocket);
+        {error, Reason} ->
+            lager:debug("[efka_tcp_server] Failed to start server: ~p~n", [Reason]),
+            exit(Reason)
+    end.
+
+main_loop(ListenSocket) ->
+    case gen_tcp:accept(ListenSocket) of
+        {ok, Socket} ->
+            % 为每个新连接生成一个处理进程
+            {ok, ChannelPid} = tcp_connection_sup:start_child(Socket),
+            ok = gen_tcp:controlling_process(Socket, ChannelPid),
+            % 继续监听下一个连接
+            main_loop(ListenSocket);
+        {error, closed} ->
+            lager:debug("[efka_tcp_server] Server socket closed"),
+            exit(tcp_closed);
+        {error, Reason} ->
+            lager:debug("[efka_tcp_server] Accept error: ~p", [Reason]),
+            exit(Reason)
+    end.

apps/efka/src/tcp_server/tcp_sup.erl

@@ -0,0 +1,72 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 26. 8月 2025 14:36
+%%%-------------------------------------------------------------------
+-module(tcp_sup).
+-author("anlicheng").
+
+-behaviour(supervisor).
+
+%% API
+-export([start_link/0]).
+
+%% Supervisor callbacks
+-export([init/1]).
+
+-define(SERVER, ?MODULE).
+
+%%%===================================================================
+%%% API functions
+%%%===================================================================
+
+%% @doc Starts the supervisor
+-spec(start_link() -> {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link() ->
+    supervisor:start_link({local, ?SERVER}, ?MODULE, []).
+
+%%%===================================================================
+%%% Supervisor callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Whenever a supervisor is started using supervisor:start_link/[2,3],
+%% this function is called by the new process to find out about
+%% restart strategy, maximum restart frequency and child
+%% specifications.
+-spec(init(Args :: term()) ->
+    {ok, {SupFlags :: {RestartStrategy :: supervisor:strategy(),
+        MaxR :: non_neg_integer(), MaxT :: non_neg_integer()},
+        [ChildSpec :: supervisor:child_spec()]}}
+    | ignore | {error, Reason :: term()}).
+init([]) ->
+    SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
+
+    Specs = [
+        #{
+            id => 'tcp_connection_sup',
+            start => {'tcp_connection_sup', start_link, []},
+            restart => permanent,
+            shutdown => 2000,
+            type => supervisor,
+            modules => ['tcp_connection_sup']
+        },
+
+        #{
+            id => 'tcp_server',
+            start => {'tcp_server', start_link, []},
+            restart => permanent,
+            shutdown => 2000,
+            type => worker,
+            modules => ['tcp_server']
+        }
+    ],
+
+    {ok, {SupFlags, Specs}}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================

apps/efka/src/tests/efka_iot_outbox_test.erl

@@ -1,315 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc
-%%% efka_iot_outbox 的手动模拟测试模块。
-%%%
-%%% 该模块基于 gen_server 实现，用两个定时 tick 模拟一边写入、一边读取：
-%%% - write_tick 每次向 efka_iot_outbox 写入一条 packet。
-%%% - read_tick 每次从 efka_iot_outbox 读取并 ack 一条 packet。
-%%% - 默认 read_interval = write_interval * 2，因此读取速度是写入速度的
-%%%   50%。
-%%% - 读取到的数据会通过 logger:debug/2 输出。
-%%%
-%%% 默认使用独立临时 dir，避免污染真实运行目录。也可以通过 start_link/1
-%%% 传入 {dir, Dir} 指定 outbox 目录。
-%%%
-%%% 示例：
-%%% <pre>
-%%% efka_iot_outbox_test:start_link([
-%%%     {dir, "/private/tmp/efka_iot_outbox_test"},
-%%%     {write_interval, 100},
-%%%     {read_interval, 200},
-%%%     {max_writes, 1000}
-%%% ]).
-%%% efka_iot_outbox_test:stats().
-%%% efka_iot_outbox_test:stop().
-%%% </pre>
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_outbox_test).
-
--behaviour(gen_server).
-
--export([start/0, start/1, start_link/0, start_link/1, stop/0, stats/0]).
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
--export([test/0]).
-
--define(SERVER, ?MODULE).
--define(DEFAULT_WRITE_INTERVAL, 100).
--define(DEFAULT_SEGMENT_RECORD_LIMIT, 2000).
--define(DEFAULT_MAX_SEGMENTS, 5).
--define(DEFAULT_MAX_RECORD_BYTES, 16 * 1024 * 1024).
-
--record(state, {
-    outbox :: efka_iot_outbox:outbox(),
-    dir :: file:filename_all(),
-    write_interval = ?DEFAULT_WRITE_INTERVAL :: pos_integer(),
-    read_interval = ?DEFAULT_WRITE_INTERVAL * 2 :: pos_integer(),
-    max_writes = infinity :: pos_integer() | infinity,
-    write_timer = undefined :: undefined | reference(),
-    read_timer = undefined :: undefined | reference(),
-    write_count = 0 :: non_neg_integer(),
-    appended_count = 0 :: non_neg_integer(),
-    read_count = 0 :: non_neg_integer(),
-    dropped_count = 0 :: non_neg_integer(),
-    error_count = 0 :: non_neg_integer()
-}).
-
--type option() ::
-    {dir, file:filename_all()} |
-    {write_interval, pos_integer()} |
-    {read_interval, pos_integer()} |
-    {max_writes, pos_integer() | infinity} |
-    {segment_record_limit, pos_integer()} |
-    {max_segments, pos_integer()} |
-    {max_record_bytes, pos_integer()}.
-
-
-test() ->
-    efka_iot_outbox_test:start_link([
-        {write_interval, 10},
-        {read_interval, 20},
-        {max_writes, 100_0000}
-    ]).
-
--spec start() -> {ok, pid()} | {error, term()}.
-start() ->
-    start([]).
-
--spec start([option()] | map()) -> {ok, pid()} | {error, term()}.
-start(Options) ->
-    gen_server:start({local, ?SERVER}, ?MODULE, Options, []).
-
--spec start_link() -> {ok, pid()} | {error, term()}.
-start_link() ->
-    start_link([]).
-
--spec start_link([option()] | map()) -> {ok, pid()} | {error, term()}.
-start_link(Options) ->
-    gen_server:start_link({local, ?SERVER}, ?MODULE, Options, []).
-
--spec stop() -> ok.
-stop() ->
-    gen_server:stop(?SERVER).
-
--spec stats() -> map().
-stats() ->
-    gen_server:call(?SERVER, stats).
-
--spec init([option()] | map()) -> {ok, #state{}} | {stop, term()}.
-init(Options0) ->
-    process_flag(trap_exit, true),
-    Options = normalize_options(Options0),
-    WriteInterval = positive_option(write_interval, Options, ?DEFAULT_WRITE_INTERVAL),
-    ReadInterval = positive_option(read_interval, Options, WriteInterval * 2),
-    SegmentRecordLimit = positive_option(segment_record_limit, Options, ?DEFAULT_SEGMENT_RECORD_LIMIT),
-    MaxSegments = positive_option(max_segments, Options, ?DEFAULT_MAX_SEGMENTS),
-    MaxRecordBytes = positive_option(max_record_bytes, Options, ?DEFAULT_MAX_RECORD_BYTES),
-    MaxWrites = max_writes_option(Options),
-
-    Dir = "/private/tmp/efka_outbox/",
-    ok = filelib:ensure_dir(filename:join(Dir, "dummy")),
-    OutboxOptions = #{
-        dir => Dir,
-        segment_record_limit => SegmentRecordLimit,
-        max_segments => MaxSegments,
-        max_record_bytes => MaxRecordBytes
-    },
-    case efka_iot_outbox:open(OutboxOptions) of
-        {ok, Outbox} ->
-            State0 = #state{
-                outbox = Outbox,
-                dir = Dir,
-                write_interval = WriteInterval,
-                read_interval = ReadInterval,
-                max_writes = MaxWrites
-            },
-            State1 = schedule_write(State0, 0),
-            State2 = schedule_read(State1, ReadInterval),
-            {ok, State2};
-        {error, Reason} ->
-            {stop, Reason}
-    end.
-
--spec handle_call(term(), gen_server:from(), #state{}) ->
-    {reply, term(), #state{}}.
-handle_call(stats, _From, State) ->
-    {reply, state_stats(State), State};
-handle_call(_Request, _From, State) ->
-    {reply, {error, unknown_call}, State}.
-
--spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
-handle_cast(_Request, State) ->
-    {noreply, State}.
-
--spec handle_info(term(), #state{}) -> {noreply, #state{}}.
-handle_info(write_tick, State0 = #state{write_timer = WriteTimer}) ->
-    State1 = State0#state{write_timer = undefined},
-    State2 = case can_write(State1) of
-        true ->
-            append_one(State1);
-        false ->
-            State1
-    end,
-    State3 = case can_write(State2) of
-        true ->
-            schedule_write(State2, State2#state.write_interval);
-        false ->
-            State2
-    end,
-    _ = WriteTimer,
-    {noreply, State3};
-handle_info(read_tick, State0 = #state{read_timer = ReadTimer}) ->
-    State1 = State0#state{read_timer = undefined},
-    State2 = read_one(State1),
-    State3 = schedule_read(State2, State2#state.read_interval),
-    _ = ReadTimer,
-    {noreply, State3};
-handle_info(_Info, State) ->
-    {noreply, State}.
-
--spec terminate(term(), #state{}) -> ok.
-terminate(_Reason, #state{outbox = Outbox, write_timer = WriteTimer, read_timer = ReadTimer}) ->
-    cancel_timer(WriteTimer),
-    cancel_timer(ReadTimer),
-    efka_iot_outbox:close(Outbox),
-    ok.
-
--spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
-code_change(_OldVsn, State, _Extra) ->
-    {ok, State}.
-
--spec append_one(#state{}) -> #state{}.
-append_one(State = #state{outbox = Outbox, write_count = WriteCount}) ->
-    PacketId = WriteCount + 1,
-    Packet = term_to_binary(#{
-        source => efka_iot_outbox_test,
-        packet_id => PacketId,
-        monotonic_time => erlang:monotonic_time(millisecond)
-    }),
-    case efka_iot_outbox:append(Packet, Outbox) of
-        {ok, NOutbox} ->
-            State#state{
-                outbox = NOutbox,
-                write_count = PacketId,
-                appended_count = State#state.appended_count + 1
-            };
-        {dropped, capacity_reached, NOutbox} ->
-            logger:warning("[efka_iot_outbox_test] outbox capacity reached, packet_id: ~p", [PacketId]),
-            State#state{
-                outbox = NOutbox,
-                write_count = PacketId,
-                dropped_count = State#state.dropped_count + 1
-            };
-        {error, Reason} ->
-            logger:warning("[efka_iot_outbox_test] append failed, packet_id: ~p, reason: ~p", [PacketId, Reason]),
-            State#state{
-                write_count = PacketId,
-                error_count = State#state.error_count + 1
-            }
-    end.
-
--spec read_one(#state{}) -> #state{}.
-read_one(State = #state{outbox = Outbox}) ->
-    case efka_iot_outbox:next(Outbox) of
-        {ok, Seq, Packet} ->
-            logger:debug("[efka_iot_outbox_test] read seq: ~p, packet: ~p", [Seq, decode_packet(Packet)]),
-            case efka_iot_outbox:ack(Seq, Outbox) of
-                {ok, NOutbox} ->
-                    State#state{
-                        outbox = NOutbox,
-                        read_count = State#state.read_count + 1
-                    };
-                {error, Reason} ->
-                    logger:warning("[efka_iot_outbox_test] ack failed, seq: ~p, reason: ~p", [Seq, Reason]),
-                    State#state{error_count = State#state.error_count + 1}
-            end;
-        eof ->
-            State;
-        {error, Reason} ->
-            logger:warning("[efka_iot_outbox_test] read failed, reason: ~p", [Reason]),
-            State#state{error_count = State#state.error_count + 1}
-    end.
-
--spec schedule_write(#state{}, non_neg_integer()) -> #state{}.
-schedule_write(State, Delay) ->
-    Ref = erlang:send_after(Delay, self(), write_tick),
-    State#state{write_timer = Ref}.
-
--spec schedule_read(#state{}, non_neg_integer()) -> #state{}.
-schedule_read(State, Delay) ->
-    Ref = erlang:send_after(Delay, self(), read_tick),
-    State#state{read_timer = Ref}.
-
--spec cancel_timer(undefined | reference()) -> ok.
-cancel_timer(undefined) ->
-    ok;
-cancel_timer(Ref) ->
-    _ = erlang:cancel_timer(Ref),
-    ok.
-
--spec can_write(#state{}) -> boolean().
-can_write(#state{max_writes = infinity}) ->
-    true;
-can_write(#state{write_count = WriteCount, max_writes = MaxWrites}) ->
-    WriteCount < MaxWrites.
-
--spec state_stats(#state{}) -> map().
-state_stats(#state{
-    dir = Dir,
-    write_interval = WriteInterval,
-    read_interval = ReadInterval,
-    max_writes = MaxWrites,
-    write_count = WriteCount,
-    appended_count = AppendedCount,
-    read_count = ReadCount,
-    dropped_count = DroppedCount,
-    error_count = ErrorCount
-}) ->
-    #{
-        dir => Dir,
-        write_interval => WriteInterval,
-        read_interval => ReadInterval,
-        max_writes => MaxWrites,
-        write_count => WriteCount,
-        appended_count => AppendedCount,
-        read_count => ReadCount,
-        dropped_count => DroppedCount,
-        error_count => ErrorCount,
-        pending_count => AppendedCount - ReadCount
-    }.
-
--spec normalize_options([option()] | map()) -> map().
-normalize_options(Options) when is_map(Options) ->
-    Options;
-normalize_options(Options) when is_list(Options) ->
-    maps:from_list(Options).
-
--spec positive_option(atom(), map(), pos_integer()) -> pos_integer().
-positive_option(Key, Options, Default) ->
-    case maps:get(Key, Options, Default) of
-        Value when is_integer(Value), Value > 0 ->
-            Value;
-        Value ->
-            error({invalid_option, Key, Value})
-    end.
-
--spec max_writes_option(map()) -> pos_integer() | infinity.
-max_writes_option(Options) ->
-    case maps:get(max_writes, Options, infinity) of
-        infinity ->
-            infinity;
-        Value when is_integer(Value), Value > 0 ->
-            Value;
-        Value ->
-            error({invalid_option, max_writes, Value})
-    end.
-
--spec decode_packet(binary()) -> term().
-decode_packet(Packet) ->
-    try binary_to_term(Packet, [safe]) of
-        Term ->
-            Term
-    catch
-        error:_ ->
-            Packet
-    end.

apps/efka/src/websocket_server/ws_channel.erl

@@ -0,0 +1,150 @@
+%%%-------------------------------------------------------------------
+%%% @author licheng5
+%%% @copyright (C) 2021, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 11. 1月 2021 上午12:17
+%%%-------------------------------------------------------------------
+-module(ws_channel).
+-author("licheng5").
+-include("efka_service.hrl").
+
+%% API
+-export([init/2]).
+-export([websocket_init/1, websocket_handle/2, websocket_info/2, terminate/3]).
+
+%% 最大的等待时间
+-define(PENDING_TIMEOUT, 10 * 1000).
+
+-record(state, {
+    packet_id = 1,
+    service_id :: undefined | binary(),
+    service_pid :: undefined | pid(),
+    is_registered = false :: boolean(),
+
+    %% 请求响应的对应关系, #{packet_id => {ReceiverPid, Ref}}; 自身的inflight需要超时逻辑处理
+    inflight = #{}
+}).
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% 逻辑处理方法
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+init(Req, Opts) ->
+    {cowboy_websocket, Req, Opts}.
+
+websocket_init(_State) ->
+    lager:debug("[ws_channel] get a new connection"),
+    %% 初始状态为true
+    {ok, #state{packet_id = 1}}.
+
+websocket_handle({binary, <<?PACKET_REQUEST, Data/binary>>}, State) ->
+    Request = jiffy:decode(Data, [return_maps]),
+    handle_request(Request, State);
+
+%% 处理micro-client:response => efka 的响应
+websocket_handle({binary, <<?PACKET_RESPONSE:8, Data/binary>>}, State = #state{inflight = Inflight}) ->
+    {PacketId, Reply} = case jiffy:decode(Data, [return_maps]) of
+                            #{<<"id">> := Id, <<"result">> := Result} ->
+                                {Id, {ok, Result}};
+                            #{<<"id">> := Id, <<"error">> := #{<<"code">> := _Code, <<"message">> := Error}} ->
+                                {Id, {error, Error}}
+                        end,
+    NInflight = gen_channel:channel_reply(PacketId, Reply, Inflight),
+    {ok, State#state{inflight = NInflight}};
+
+websocket_handle(Info, State) ->
+    lager:error("[ws_channel] get a unknown message: ~p, channel will closed", [Info]),
+    {stop, State}.
+
+websocket_info({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{packet_id = PacketId, inflight = Inflight}) ->
+    PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"push_config">>, <<"params">> => #{<<"config">> => ConfigJson}},
+    Packet = jiffy:encode(PushConfig, [force_utf8]),
+    Reply = <<?PACKET_PUSH:8, Packet/binary>>,
+    erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}),
+
+    {reply, {binary, Reply}, State#state{packet_id = gen_channel:next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}};
+
+%% 远程调用
+websocket_info({invoke, Ref, ReceiverPid, Payload}, State = #state{packet_id = PacketId, inflight = Inflight}) ->
+    PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"invoke">>, <<"params">> => #{<<"payload">> => Payload}},
+    Packet = jiffy:encode(PushConfig, [force_utf8]),
+    Reply = <<?PACKET_PUSH:8, Packet/binary>>,
+    erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}),
+
+    {reply, {binary, Reply}, State#state{packet_id = gen_channel:next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}};
+
+%% 订阅的消息
+websocket_info({topic_broadcast, Topic, Content}, State = #state{}) ->
+    Packet = jiffy:encode(#{<<"topic">> => Topic, <<"content">> => Content}, [force_utf8]),
+    Reply = <<?PACKET_PUB:8, Packet/binary>>,
+    {reply, {binary, Reply}, State};
+
+%% 超时逻辑处理
+websocket_info({timeout, _, {pending_timeout, Id}}, State = #state{inflight = Inflight}) ->
+    NInflight = gen_channel:channel_reply(Id, {error, <<"timeout">>}, Inflight),
+    {ok, State#state{inflight = NInflight}};
+
+%% service进程关闭
+websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) ->
+    lager:debug("[tcp_channel] service_pid: ~p, exited: ~p", [ServicePid, Reason]),
+    {stop, State#state{service_pid = undefined}};
+
+websocket_info({timeout, _, {stop, Reason}}, State) ->
+    lager:debug("[ws_channel] the channel will be closed with reason: ~p", [Reason]),
+    {stop, State};
+%% 处理关闭信号
+websocket_info({stop, Reason}, State) ->
+    lager:debug("[ws_channel] the channel will be closed with reason: ~p", [Reason]),
+    {stop, State};
+
+%% 处理其他未知消息
+websocket_info(Info, State) ->
+    lager:debug("[ws_channel] channel get unknown info: ~p", [Info]),
+    {ok, State}.
+
+%% 进程关闭事件
+terminate(Reason, _Req, State) ->
+    lager:debug("[ws_channel] channel close with reason: ~p, state is: ~p", [Reason, State]),
+    ok.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% helper methods
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+%% 注册
+handle_request(#{<<"id">> := Id, <<"method">> := <<"register">>, <<"params">> := #{<<"service_id">> := ServiceId}}, State) ->
+    case gen_channel:register(Id, ServiceId) of
+        {error, Reply} ->
+            delay_stop(10, normal),
+            {reply, {binary,  <<?PACKET_RESPONSE:8, Reply/binary>>}, State};
+        {ok, Reply, ServicePid} ->
+            delay_stop(10, normal),
+            {reply, {binary, <<?PACKET_RESPONSE:8, Reply/binary>>}, State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}}
+    end;
+%% 请求参数
+handle_request(#{<<"id">> := Id, <<"method">> := <<"request_config">>}, State = #state{service_pid = ServicePid, is_registered = true}) ->
+    {ok, ConfigJson} = efka_service:request_config(ServicePid),
+    Packet = gen_channel:json_result(Id, ConfigJson),
+    Reply = <<?PACKET_RESPONSE:8, Packet/binary>>,
+    {reply, {binary, Reply}, State};
+
+%% 数据项
+handle_request(#{<<"id">> := 0, <<"method">> := <<"metric_data">>,
+    <<"params">> := #{<<"device_uuid">> := DeviceUUID, <<"route_key">> := RouteKey, <<"metric">> := Metric}}, State = #state{service_pid = ServicePid, is_registered = true}) ->
+    efka_service:metric_data(ServicePid, DeviceUUID, RouteKey, Metric),
+    {ok, State};
+
+%% Event事件
+handle_request(#{<<"id">> := 0, <<"method">> := <<"event">>, <<"params">> := #{<<"event_type">> := EventType, <<"body">> := Body}}, State = #state{service_pid = ServicePid, is_registered = true}) ->
+    efka_service:send_event(ServicePid, EventType, Body),
+    {ok, State};
+
+%% 订阅事件
+handle_request(#{<<"id">> := 0, <<"method">> := <<"subscribe">>, <<"params">> := #{<<"topic">> := Topic}}, State = #state{is_registered = true}) ->
+    efka_subscription:subscribe(Topic, self()),
+    {ok, State}.
+
+delay_stop(Timeout, Reason) when is_integer(Timeout) ->
+    erlang:start_timer(Timeout, self(), {stop, Reason}).

apps/efka/src/websocket_server/ws_server.erl

@@ -0,0 +1,34 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 26. 8月 2025 14:28
+%%%-------------------------------------------------------------------
+-module(ws_server).
+-author("anlicheng").
+
+%% API
+-export([start_server/0]).
+
+start_server() ->
+    {ok, Props} = application:get_env(efka, ws_server),
+    Acceptors = proplists:get_value(acceptors, Props, 50),
+    MaxConnections = proplists:get_value(max_connections, Props, 10240),
+    Backlog = proplists:get_value(backlog, Props, 1024),
+    Port = proplists:get_value(port, Props),
+
+    Dispatcher = cowboy_router:compile([
+        {'_', [{"/ws", ws_channel, []}]}
+    ]),
+
+    TransOpts = [
+        {port, Port},
+        {num_acceptors, Acceptors},
+        {backlog, Backlog},
+        {max_connections, MaxConnections}
+    ],
+    {ok, Pid} = cowboy:start_clear(ws_listener, TransOpts, #{env => #{dispatch => Dispatcher}}),
+
+    lager:debug("[efka_app] websocket server start at: ~p, pid is: ~p", [Port, Pid]).

config/sys.config

@@ -0,0 +1,67 @@
+[
+  {efka, [
+    {root_dir, "/usr/local/code/efka"},
+
+    {dets_dir, "/usr/local/code/tmp/dets/"},
+
+    {tcp_server, [
+      {port, 18088}
+    ]},
+
+    {ws_server, [
+      {port, 18080},
+      {acceptors, 10},
+      {max_connections, 1024},
+      {backlog, 256}
+    ]},
+
+    {tls_server, [
+      {host, "localhost"},
+      {port, 443}
+    ]},
+
+    {auth, [
+      {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"},
+      {username, "test"},
+      {salt, "salt2345"},
+      {token, "token124"}
+    ]}
+
+  ]},
+
+  %% 系统日志配置，系统日志为lager, 支持日志按日期自动分割
+  {lager, [
+    {colored, true},
+    %% Whether to write a crash log, and where. Undefined means no crash logger.
+    {crash_log, "trade_hub.crash.log"},
+    %% Maximum size in bytes of events in the crash log - defaults to 65536
+    {crash_log_msg_size, 65536},
+    %% Maximum size of the crash log in bytes, before its rotated, set
+    %% to 0 to disable rotation - default is 0
+    {crash_log_size, 10485760},
+    %% What time to rotate the crash log - default is no time
+    %% rotation. See the README for a description of this format.
+    {crash_log_date, "$D0"},
+    %% Number of rotated crash logs to keep, 0 means keep only the
+    %% current one - default is 0
+    {crash_log_count, 5},
+    %% Whether to redirect error_logger messages into lager - defaults to true
+    {error_logger_redirect, true},
+
+    %% How big the gen_event mailbox can get before it is switched into sync mode
+    {async_threshold, 20},
+    %% Switch back to async mode, when gen_event mailbox size decrease from `async_threshold'
+    %% to async_threshold - async_threshold_window
+    {async_threshold_window, 5},
+
+    {handlers, [
+      %% debug | info | warning | error, 日志级别
+      {lager_console_backend, debug},
+      {lager_file_backend, [{file, "debug.log"}, {level, debug}, {size, 314572800}]},
+      {lager_file_backend, [{file, "notice.log"}, {level, notice}, {size, 314572800}]},
+      {lager_file_backend, [{file, "error.log"}, {level, error}, {size, 314572800}]},
+      {lager_file_backend, [{file, "info.log"}, {level, info}, {size, 314572800}]}
+    ]}
+
+  ]}
+].

config/sys.config.src

@@ -1,70 +0,0 @@
-[
-  {efka, [
-    {dets_dir, "${EFKA_DETS_DIR}"},
-
-    {websocket_server, [
-      {port, 18080},
-      {acceptors, 10},
-      {max_connections, 1024},
-      {backlog, 256}
-    ]},
-
-    {iot_server, [
-      {host, "${EFKA_IOT_HOST}"},
-      {tls_port, 1443},
-      {udp_port, 24000}
-    ]},
-
-    {stream_targets, [
-      {1, [
-        {name, manager},
-        {host, "127.0.0.1"},
-        {port, 81},
-        {connect_timeout, 3000},
-        {idle_timeout, 120000}
-      ]}
-    ]},
-
-    {heartbeat, [
-      {interval, 5000}
-    ]},
-
-    {auth, [
-      {uuid, "${EFKA_AUTH_UUID}"},
-      {token, "${EFKA_AUTH_TOKEN}"}
-    ]}
-
-  ]},
-
-  {docker, [
-    {root_dir, "${EFKA_DOCKER_ROOT_DIR}"}
-  ]},
-
-  %% 系统日志配置，使用 OTP logger
-  {kernel, [
-    %% 设置 Logger 的 primary log level
-    {logger_level, debug},
-    {logger, [
-      {handler, default, logger_std_h,
-        #{
-          level => debug,
-          formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}}
-        }
-      },
-
-      {handler, disk, logger_disk_log_h,
-        #{
-          level => debug,
-          config => #{
-            file => "log/debug.log",
-            max_no_files => 10,
-            max_no_bytes => 524288000
-          },
-          formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}}
-        }
-      }
-
-    ]}
-  ]}
-
-].

config/vm.args

@@ -1,9 +1,9 @@
--sname efka
+-name efka
 
 -setcookie efka_cookie
 -kernel start_group false
 
--mnesia dir '"${EFKA_MNESIA_DIR}"'
+-mnesia dir '"/usr/local/var/mnesia/efka"'
 -mnesia dump_log_write_threshold 5000
 -mnesia dc_dump_limit 40
 

docs/container_command_docker_json.md

@@ -1,803 +0,0 @@
-# IOT 容器命令到 Docker JSON 的转换说明
-
-本文档描述 `efka` 收到 `iot` 下发的容器管理 command 后，接受的 Erlang map 格式，以及 deploy 命令中 `create` 参数如何转换成 Docker Engine API 接收的 JSON。
-
-对应代码：
-
-- command 接收入口：[src/iot/efka_iot_client.erl](/usr/local/code/cloudkit/efka/apps/efka/src/iot/efka_iot_client.erl:177)
-- 部署任务管理：[docker_deploy_manager.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deploy_manager.erl:36)
-- 部署执行：[docker_deployer.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deployer.erl:39)
-- Docker JSON 构造：[docker_container_builder.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_container_builder.erl:14)
-- Docker API 调用：[docker_commands.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_commands.erl:36)
-
-## 1. 协议入口
-
-`iot` 通过 TLS 长连接向 `efka` 下发容器命令：
-
-```erlang
-{<<"command">>, Ref, {<<"container">>, CommandMap}}
-```
-
-`efka` 执行后回复：
-
-```erlang
-{<<"command_response">>, Ref, {<<"container">>, Reply}}
-```
-
-`Reply` 取值：
-
-```erlang
-<<"ok">>
-{<<"ok">>, Result}
-{<<"error">>, Reason}
-```
-
-`Ref` 是 `crypto:strong_rand_bytes(16)` 生成的 16 字节 binary。网络帧只使用 `binary_to_term(PacketBin, [safe])` 可解码的 safe term；协议 label、业务 label、map key 和 action 都使用 binary。
-
-只有 `efka_iot_client` 处于 `activated` 状态时，容器命令才会正常执行；处于非 activated 状态时会返回错误。
-
-## 2. 容器命令 map
-
-`efka_iot_client` 收到网络协议里的 binary-key `CommandMap` 后，直接按 binary key 和 binary action 做函数参数匹配，不再做整包 atom-key 转换。
-
-- command 顶层、`target`、deploy `params`、`create` 中间结构都使用 binary key。
-- `<<"action">>` 的取值使用 binary：`<<"list">>`、`<<"deploy">>`、`<<"start">>`、`<<"stop">>`、`<<"kill">>`、`<<"remove">>`、`<<"config">>`。
-- `undefined` 作为 Erlang 已有 atom 可以直接出现在 safe term 中；协议不再使用额外的哨兵 binary 表示缺省值。
-
-下面各小节描述的是 `efka` 直接接收和处理的 map 格式。
-
-### list
-
-```erlang
-#{<<"action">> => <<"list">>}
-```
-
-执行：
-
-```erlang
-docker_commands:get_containers()
-```
-
-Docker API：
-
-```http
-GET /containers/json?all=true
-```
-
-### deploy
-
-```erlang
-#{
-    <<"action">> => <<"deploy">>,
-    <<"task_id">> => TaskId,
-    <<"params">> => Params
-}
-```
-
-执行：
-
-```erlang
-docker_deploy_manager:deploy(TaskId, Params)
-```
-
-`deploy/2` 会启动独立部署进程，HTTP command response 只表示部署任务是否成功启动。实际部署过程和结果通过 `task_event` 消息流上报给 `iot`。
-
-### start
-
-```erlang
-#{
-    <<"action">> => <<"start">>,
-    <<"target">> => Target
-}
-```
-
-执行：
-
-```erlang
-docker_commands:start_container(ContainerNameOrId)
-```
-
-Docker API：
-
-```http
-POST /containers/{name_or_id}/start
-```
-
-### stop
-
-```erlang
-#{
-    <<"action">> => <<"stop">>,
-    <<"target">> => Target,
-    <<"timeout_seconds">> => TimeoutSeconds
-}
-```
-
-执行：
-
-```erlang
-docker_commands:stop_container(ContainerNameOrId, TimeoutSeconds)
-```
-
-Docker API：
-
-```http
-POST /containers/{name_or_id}/stop?t={TimeoutSeconds}
-```
-
-### kill
-
-```erlang
-#{
-    <<"action">> => <<"kill">>,
-    <<"target">> => Target,
-    <<"signal">> => Signal
-}
-```
-
-执行：
-
-```erlang
-docker_commands:kill_container(ContainerNameOrId, Signal)
-```
-
-### remove
-
-```erlang
-#{
-    <<"action">> => <<"remove">>,
-    <<"target">> => Target,
-    <<"force">> => Force,
-    <<"remove_volumes">> => RemoveVolumes
-}
-```
-
-执行：
-
-```erlang
-docker_commands:remove_container(ContainerNameOrId, Force, RemoveVolumes)
-```
-
-### config
-
-```erlang
-#{
-    <<"action">> => <<"config">>,
-    <<"target">> => Target,
-    <<"config">> => Config
-}
-```
-
-执行：
-
-```erlang
-docker_helper:update_container_config(ContainerNameOrId, iolist_to_binary(Config))
-```
-
-该命令不会调用 Docker API，只会更新 `efka` 主机上对应容器目录里的 `service.conf`。
-
-## 3. Target 解析规则
-
-`target` 是 map：
-
-```erlang
-#{
-    <<"name">> => ContainerName,
-    <<"id">> => ContainerId
-}
-```
-
-解析规则：
-
-- 优先使用 `name`。
-- `name` 为空时使用 `id`。
-- `name` 和 `id` 都为空时会触发匹配错误。
-
-## 4. deploy params 格式
-
-`deploy` 的 `params` 必须包含：
-
-```erlang
-#{
-    <<"container_name">> => ContainerName,
-    <<"create">> => Create
-}
-```
-
-字段说明：
-
-| 字段 | 类型 | 说明 |
-| --- | --- | --- |
-| `container_name` | binary | 容器名称。用于容器创建 URL 的 `name` 参数，也会注入环境变量 `CONTAINER_NAME`。 |
-| `create` | map | Docker create options 的中间结构，由 `iot` 侧构造，`efka` 侧补丁后转为 Docker JSON。 |
-
-`docker_deploy_manager` 会根据 `container_name` 按系统默认规则确保容器目录存在。目录固定为：
-
-```text
-docker.root_dir/container_name/
-```
-
-即使旧客户端在 `params` 中携带 `container_dir`，当前 deploy 流程也会忽略它，避免 HTTP 调用方控制 efka 主机上的写入目录。
-
-## 5. create 中间结构
-
-`create` 的结构：
-
-```erlang
-#{
-    <<"config">> => ContainerConfig,
-    <<"host_config">> => HostConfig,
-    <<"networking_config">> => NetworkingConfig
-}
-```
-
-这三个 map 会被 `docker_container_builder:build_options/3` 转成 Docker Engine API JSON。
-
-## 6. efka 自动补丁规则
-
-在转 Docker JSON 前，`efka` 会先执行补丁：
-
-### 环境变量
-
-向 `create.config.env` 前置注入：
-
-```erlang
-<<"CONTAINER_NAME=", ContainerName/binary>>
-```
-
-如果原 env 列表已经包含完全相同的值，则不会重复添加。
-
-### 配置文件 volume
-
-向 `create.config.volumes` 前置注入容器内配置路径：
-
-```erlang
-<<"/usr/local/etc/service.conf">>
-```
-
-### 配置文件 bind
-
-向 `create.host_config.binds` 前置注入宿主机配置文件映射：
-
-```erlang
-<<ConfigFile/binary, ":/usr/local/etc/service.conf">>
-```
-
-其中 `ConfigFile` 是当前容器目录下的：
-
-```text
-service.conf
-```
-
-如果列表里已存在完全相同的 bind，则不会重复添加。
-
-## 7. Docker JSON 顶层结构
-
-`docker_commands:create_container/2` 最终调用：
-
-```http
-POST /containers/create?name={container_name}
-Content-Type: application/json
-```
-
-请求 body 来自：
-
-```erlang
-Options = docker_container_builder:build_options(ContainerName, ContainerDir, Create),
-Body = iolist_to_binary(json:encode(Options))
-```
-
-最终 JSON 顶层字段：
-
-```json
-{
-  "Image": "...",
-  "Cmd": [],
-  "Entrypoint": [],
-  "Env": [],
-  "Labels": {},
-  "Volumes": {},
-  "User": "",
-  "WorkingDir": "",
-  "Hostname": "",
-  "ExposedPorts": {},
-  "NetworkingConfig": {},
-  "Healthcheck": {},
-  "HostConfig": {}
-}
-```
-
-空列表或空 map 字段有的会保留，有的会被省略到 `{}`，具体见下面映射规则。
-
-## 8. create.config 到 Docker JSON
-
-| 中间字段 | Docker JSON 字段 | 转换规则 |
-| --- | --- | --- |
-| `image` | `Image` | 转成 binary；缺省为 `""`。部署流程拉镜像时会对 image 补 `:latest`，但 create JSON 使用传入值。 |
-| `cmd` | `Cmd` | 列表元素逐个转 binary。 |
-| `entrypoint` | `Entrypoint` | 列表元素逐个转 binary。 |
-| `env` | `Env` | 列表元素逐个转 binary，并自动注入 `CONTAINER_NAME=...`。 |
-| `labels` | `Labels` | key/value 都转 binary。 |
-| `volumes` | `Volumes` | 转成 Docker 要求的 object，key 是容器内路径，value 是 `{}`。 |
-| `user` | `User` | 转成 binary；缺省为 `""`。 |
-| `working_dir` | `WorkingDir` | 转成 binary；缺省为 `""`。 |
-| `hostname` | `Hostname` | 转成 binary；缺省为 `""`。 |
-| `exposed_ports` | `ExposedPorts` | 转成 Docker 端口 object。 |
-| `healthcheck` | `Healthcheck` | 转成 Docker Healthcheck object；未传时为 `{}`。 |
-
-### Volumes
-
-输入：
-
-```erlang
-[<<"/data">>, <<"/usr/local/etc/service.conf">>]
-```
-
-输出 JSON：
-
-```json
-{
-  "Volumes": {
-    "/data": {},
-    "/usr/local/etc/service.conf": {}
-  }
-}
-```
-
-### ExposedPorts
-
-输入：
-
-```erlang
-[
-    #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-    #{<<"container_port">> => 53, <<"protocol">> => <<"udp">>}
-]
-```
-
-输出 JSON：
-
-```json
-{
-  "ExposedPorts": {
-    "80/tcp": {},
-    "53/udp": {}
-  }
-}
-```
-
-规则：
-
-- `protocol` 为空或 `<<"tcp">>` 时输出 `tcp`。
-- 其他 protocol 原样输出。
-
-### Healthcheck
-
-该字段由 iot 校验后下发。`test` 必须是 binary list，`interval_ns`、`timeout_ns` 和 `retries` 必须是非负整数；efka 只按收到的中间 map 转成 Docker JSON 字段。
-
-输入：
-
-```erlang
-#{
-    <<"test">> => [<<"CMD-SHELL">>, <<"curl -f http://localhost || exit 1">>],
-    <<"interval_ns">> => 30000000000,
-    <<"timeout_ns">> => 10000000000,
-    <<"retries">> => 3
-}
-```
-
-输出 JSON：
-
-```json
-{
-  "Healthcheck": {
-    "Test": ["CMD-SHELL", "curl -f http://localhost || exit 1"],
-    "Interval": 30000000000,
-    "Timeout": 10000000000,
-    "Retries": 3
-  }
-}
-```
-
-## 9. create.host_config 到 Docker JSON
-
-`host_config` 会被合并到 Docker JSON 的 `HostConfig` 字段。
-
-| 中间字段 | Docker JSON 字段 | 转换规则 |
-| --- | --- | --- |
-| `binds` | `HostConfig.Binds` | 列表元素逐个转 binary，并自动注入 `service.conf` bind。空列表时省略。 |
-| `network_mode` | `HostConfig.NetworkMode` | 空 binary 时省略。 |
-| `restart_policy` | `HostConfig.RestartPolicy` | 转成 `Name` 和可选 `MaximumRetryCount`。 |
-| `privileged` | `HostConfig.Privileged` | 只有 true 时输出。false 时省略。 |
-| `cap_add` / `cap_drop` | `HostConfig.CapAdd` / `HostConfig.CapDrop` | 两者都为空时省略；只要一个非空，两个字段都会输出。 |
-| `devices` | `HostConfig.Devices` | 转成 Docker device object 列表。 |
-| `memory` | `HostConfig.Memory` | 0 时省略。 |
-| `memory_reservation` | `HostConfig.MemoryReservation` | 0 时省略。 |
-| `nano_cpus` | `HostConfig.NanoCpus` | 0 时省略。 |
-| `cpu_shares` | `HostConfig.CpuShares` | 0 时省略。 |
-| `port_bindings` | `HostConfig.PortBindings` | 转成 Docker 端口绑定 object；空列表时省略。 |
-| `ulimits` | `HostConfig.Ulimits` | 空列表时省略。 |
-| `tmpfs` | `HostConfig.Tmpfs` | 空 map 时省略。 |
-| `sysctls` | `HostConfig.Sysctls` | 空 map 时省略。 |
-| `extra_hosts` | `HostConfig.ExtraHosts` | 空列表时省略。 |
-
-### Binds
-
-输入：
-
-```erlang
-[
-    <<"/host/data:/data">>,
-    <<"/host/log:/var/log:ro">>
-]
-```
-
-efka 自动补丁后输出：
-
-```json
-{
-  "HostConfig": {
-    "Binds": [
-      "/path/to/container/service.conf:/usr/local/etc/service.conf",
-      "/host/data:/data",
-      "/host/log:/var/log:ro"
-    ]
-  }
-}
-```
-
-### RestartPolicy
-
-输入：
-
-```erlang
-#{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "RestartPolicy": {
-      "Name": "always"
-    }
-  }
-}
-```
-
-输入：
-
-```erlang
-#{<<"name">> => <<"on-failure">>, <<"maximum_retry_count">> => 3}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "RestartPolicy": {
-      "Name": "on-failure",
-      "MaximumRetryCount": 3
-    }
-  }
-}
-```
-
-### Devices
-
-输入：
-
-```erlang
-[
-    #{
-        <<"path_on_host">> => <<"/dev/ttyUSB0">>,
-        <<"path_in_container">> => <<"/dev/ttyUSB0">>,
-        <<"cgroup_permissions">> => <<"rwm">>
-    }
-]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Devices": [
-      {
-        "PathOnHost": "/dev/ttyUSB0",
-        "PathInContainer": "/dev/ttyUSB0",
-        "CgroupPermissions": "rwm"
-      }
-    ]
-  }
-}
-```
-
-`cgroup_permissions` 为空时默认输出 `rwm`。
-
-### PortBindings
-
-输入：
-
-```erlang
-[
-    #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-    #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
-]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "PortBindings": {
-      "80/tcp": [
-        {"HostIp": "", "HostPort": "8080"}
-      ],
-      "443/tcp": [
-        {"HostIp": "", "HostPort": "443"}
-      ]
-    }
-  }
-}
-```
-
-规则：
-
-- key 使用容器端口和协议组成：`container_port/protocol`。
-- `protocol` 为空或 `tcp` 时输出 `tcp`。
-- `HostPort` 按 Docker API 要求输出为字符串。
-
-### 资源限制
-
-输入：
-
-```erlang
-#{
-    <<"memory">> => 536870912,
-    <<"memory_reservation">> => 268435456,
-    <<"nano_cpus">> => 1500000000,
-    <<"cpu_shares">> => 512
-}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Memory": 536870912,
-    "MemoryReservation": 268435456,
-    "NanoCpus": 1500000000,
-    "CpuShares": 512
-  }
-}
-```
-
-值为 0 的资源字段会被省略。
-
-### Ulimits
-
-输入：
-
-```erlang
-[
-    #{<<"name">> => <<"nofile">>, <<"soft">> => 1024, <<"hard">> => 2048}
-]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Ulimits": [
-      {
-        "Name": "nofile",
-        "Soft": 1024,
-        "Hard": 2048
-      }
-    ]
-  }
-}
-```
-
-### Tmpfs
-
-输入：
-
-```erlang
-#{
-    <<"/tmp">> => <<>>,
-    <<"/run">> => <<"rw,size=64m">>
-}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Tmpfs": {
-      "/tmp": "",
-      "/run": "rw,size=64m"
-    }
-  }
-}
-```
-
-### Sysctls
-
-输入：
-
-```erlang
-#{<<"net.ipv4.ip_forward">> => <<"1">>}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Sysctls": {
-      "net.ipv4.ip_forward": "1"
-    }
-  }
-}
-```
-
-### ExtraHosts
-
-输入：
-
-```erlang
-[<<"host.docker.internal:host-gateway">>]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "ExtraHosts": ["host.docker.internal:host-gateway"]
-  }
-}
-```
-
-## 10. create.networking_config 到 Docker JSON
-
-输入：
-
-```erlang
-#{
-    <<"endpoints">> => [
-        #{<<"name">> => <<"bridge">>},
-        #{<<"name">> => <<"mynet">>}
-    ]
-}
-```
-
-输出：
-
-```json
-{
-  "NetworkingConfig": {
-    "EndpointsConfig": {
-      "bridge": {},
-      "mynet": {}
-    }
-  }
-}
-```
-
-未传或 endpoints 为空时输出：
-
-```json
-{
-  "NetworkingConfig": {}
-}
-```
-
-## 11. 完整转换示例
-
-收到的 deploy command：
-
-```erlang
-{<<"command">>, Ref, {<<"container">>, #{
-    <<"action">> => <<"deploy">>,
-    <<"task_id">> => 1001,
-    <<"params">> => #{
-        <<"container_name">> => <<"my_nginx">>,
-        <<"create">> => #{
-            <<"config">> => #{
-                <<"image">> => <<"docker.io/library/nginx:latest">>,
-                <<"cmd">> => [<<"nginx">>, <<"-g">>, <<"daemon off;">>],
-                <<"env">> => [<<"ENV=prod">>],
-                <<"volumes">> => [<<"/data">>],
-                <<"exposed_ports">> => [#{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>}]
-            },
-            <<"host_config">> => #{
-                <<"binds">> => [<<"/host/data:/data">>],
-                <<"restart_policy">> => #{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0},
-                <<"memory">> => 536870912,
-                <<"port_bindings">> => [
-                    #{
-                        <<"host_ip">> => <<>>,
-                        <<"host_port">> => 8080,
-                        <<"container_port">> => 80,
-                        <<"protocol">> => <<"tcp">>
-                    }
-                ]
-            },
-            <<"networking_config">> => #{
-                <<"endpoints">> => [#{<<"name">> => <<"bridge">>}]
-            }
-        }
-    }
-}}}
-```
-
-生成的 Docker JSON 示意：
-
-```json
-{
-  "Image": "docker.io/library/nginx:latest",
-  "Cmd": ["nginx", "-g", "daemon off;"],
-  "Entrypoint": [],
-  "Env": ["CONTAINER_NAME=my_nginx", "ENV=prod"],
-  "Labels": {},
-  "Volumes": {
-    "/usr/local/etc/service.conf": {},
-    "/data": {}
-  },
-  "User": "",
-  "WorkingDir": "",
-  "Hostname": "",
-  "ExposedPorts": {
-    "80/tcp": {}
-  },
-  "NetworkingConfig": {
-    "EndpointsConfig": {
-      "bridge": {}
-    }
-  },
-  "Healthcheck": {},
-  "HostConfig": {
-    "Binds": [
-      "/efka/root/my_nginx/service.conf:/usr/local/etc/service.conf",
-      "/host/data:/data"
-    ],
-    "RestartPolicy": {
-      "Name": "always"
-    },
-    "Memory": 536870912,
-    "PortBindings": {
-      "80/tcp": [
-        {
-          "HostIp": "",
-          "HostPort": "8080"
-        }
-      ]
-    }
-  }
-}
-```
-
-## 12. 部署流程补充
-
-deploy action 的执行流程：
-
-1. 根据 `container_name` 和 `docker.root_dir` 确保默认容器目录存在。
-2. 启动独立部署进程。
-3. 部署进程上报任务事件流。
-4. 规范化镜像名：如果镜像最后一段没有 tag，则补 `:latest`。
-5. 调用 Docker API 拉取镜像。
-6. 构造 Docker create JSON。
-7. 调用 `POST /containers/create?name={container_name}`。
-8. 创建空 `service.conf` 文件。
-9. 写入部署摘要日志并关闭任务事件流。
-
-注意：当前 `ensure_container_absent/2` 只上报“开始创建容器”，不会删除已有同名容器；如果 Docker 返回名称冲突，部署会失败并上报“本地容器已经存在”。

docs/efka_iot_protocol.md

@@ -1,320 +0,0 @@
-# EFKA 与 IOT 交互协议
-
-本文档描述 `efka` 与 `iot` 之间的 TLS 长连接协议。当前协议由 Erlang term 直接序列化，发送端使用 `term_to_binary/1`，接收端使用 `binary_to_term(PacketBin, [safe])`。
-
-协议帧只使用 safe external term：顶层 label、业务 label、map key 使用 binary；`Ref` 使用 `crypto:strong_rand_bytes(16)` 生成，是 16 字节 binary。网络协议里不发送 Erlang `reference()`，也不依赖动态创建 atom。
-
-## 传输层
-
-- `efka` 作为 TLS client 连接 `iot`。
-- `iot` 作为 TLS server 接收多个 `efka` 连接，一个连接对应一个 `ssl_channel` 进程。
-- socket 使用 `{packet, 4}`，每个 Erlang term binary 作为一个完整包发送。
-- `Ref` 使用 `crypto:strong_rand_bytes(16)` 生成，只在当前连接的 inflight 表内匹配。
-
-## 顶层帧
-
-协议顶层 tuple 用来表达交互语义：
-
-```erlang
-{<<"request">>, Ref, Body}
-{<<"response">>, Ref, Reply}
-{<<"command">>, Ref, {Domain, Payload}}
-{<<"command_response">>, Ref, {Domain, Reply}}
-{<<"message">>, Body}
-{<<"stream">>, StreamId, Body}
-```
-
-语义说明：
-
-| 帧 | 方向 | 语义 |
-| --- | --- | --- |
-| `{<<"request">>, Ref, Body}` | efka -> iot | efka 发起请求，需要 iot 回复 |
-| `{<<"response">>, Ref, Reply}` | iot -> efka | iot 对 efka request 的回复 |
-| `{<<"command">>, Ref, {Domain, Payload}}` | iot -> efka | iot 下发命令，需要 efka 回复 |
-| `{<<"command_response">>, Ref, {Domain, Reply}}` | efka -> iot | efka 对 iot command 的回复 |
-| `{<<"message">>, Body}` | 双向 | 异步消息，不要求回复 |
-| `{<<"stream">>, StreamId, Body}` | 双向 | 透明 TCP 字节流多路复用 |
-
-`command` 和 `command_response` 的 `Domain` 表示业务域，目前支持：
-
-- `<<"container">>`
-
-## 透明 TCP Stream
-
-`stream` 用于在同一条 TLS 长连接上复用多条透明 TCP 字节流。旧的 `request`、`response`、`command`、`command_response`、`message` 帧格式保持不变；`StreamId = 0` 只作为保留概念，不出现在新的 `stream` 帧里。
-
-`StreamId` 规则：
-
-- `0`：保留给现有控制流。
-- 奇数：`iot` 发起。
-- 偶数：`efka` 发起，当前预留。
-- 同一条 TLS 连接内 `StreamId` 不复用。
-
-`Body` 取值：
-
-```erlang
-<<"open">>
-<<"opened">>
-{<<"open_error">>, Reason}
-{<<"data">>, Chunk}
-<<"fin">>
-{<<"reset">>, Reason}
-```
-
-语义：
-
-- `open`：发起方请求打开一个透明 TCP stream。`open` 不携带参数；`efka` 端固定连接本机 `stream_target`，通常指向本机 manager/nginx 入口。
-- `opened`：本地 TCP 连接建立成功。
-- `open_error`：本地 TCP 连接建立失败，stream 结束。
-- `data`：透明字节数据，`Chunk` 是 binary。HTTP 请求头、请求体、响应头、响应体都只是该 binary 的内容，iot/efka 中间协议不解析 HTTP。
-- `fin`：半关闭，发送方后续不再发送 `data`，但仍可继续接收对端 `data`。
-- `reset`：异常关闭，双方应立即释放该 `StreamId` 的资源。
-
-典型 iot 发起访问 efka 本机监控服务的流程：
-
-```erlang
-iot  -> efka: {<<"stream">>, 1, <<"open">>}
-efka -> iot:  {<<"stream">>, 1, <<"opened">>}
-
-iot  -> efka: {<<"stream">>, 1, {<<"data">>, RawHttpRequestBytes}}
-iot  -> efka: {<<"stream">>, 1, <<"fin">>}
-
-efka -> iot:  {<<"stream">>, 1, {<<"data">>, RawHttpResponseBytes}}
-efka -> iot:  {<<"stream">>, 1, <<"fin">>}
-```
-
-## 鉴权请求
-
-初始连接由 `efka` 发起鉴权 request。每条 TLS 连接只允许一次鉴权；`iot` 侧鉴权成功后会在 `ssl_channel` 标记该连接已鉴权，如果同一连接再次发送 `auth_request`，`iot` 会直接关闭连接。
-
-```erlang
-{<<"request">>, Ref, {<<"auth_request">>, #{
-    <<"uuid">> => UUID,
-    <<"token">> => Token,
-    <<"timestamp">> => Timestamp
-}}}
-```
-
-`iot` 回复：
-
-```erlang
-{<<"response">>, Ref, {<<"auth_response">>, <<"ok">>}}
-{<<"response">>, Ref, {<<"auth_response">>, {<<"error">>, {<<"failed">>, Reason}}}}
-```
-
-处理语义：
-
-- `<<"ok">>`：`efka` 进入 `activated` 状态。
-- `{<<"error">>, {<<"failed">>, Reason}}`：鉴权失败，`iot` 返回失败响应后关闭连接；`efka` 进入重连流程。
-
-## 授权控制
-
-`/host/activate` 只修改 `iot` 本地和持久化的 host 授权状态，不再向 `efka` 下发 auth command。`efka` 可以继续保持连接并发送数据，是否处理这些数据由 `iot_host` 当前状态决定。
-
-因此当前协议没有 `{<<"command">>, Ref, {<<"auth">>, ...}}` 和 `{<<"command_response">>, Ref, {<<"auth">>, ...}}`。授权关闭时，`iot_host` 保持 channel 在线，但不处理上报数据；授权重新打开后，已在线的 channel 可以继续使用。
-
-## 容器管理命令
-
-`iot` 对 `efka` 的容器管理使用 command 语义：
-
-```erlang
-{<<"command">>, Ref, {<<"container">>, CommandMap}}
-```
-
-`efka` 回复：
-
-```erlang
-{<<"command_response">>, Ref, {<<"container">>, Reply}}
-```
-
-`Reply` 取值：
-
-```erlang
-<<"ok">>
-{<<"ok">>, Result}
-{<<"error">>, Reason}
-```
-
-`CommandMap` 使用 binary key 和 binary action；`efka` 接收后直接按 binary key/action 匹配，Docker 参数链路继续使用 binary-key map，不再转换成 atom-key map。
-
-### list
-
-```erlang
-#{<<"action">> => <<"list">>}
-```
-
-返回当前 `efka` 主机上的容器列表。
-
-### deploy
-
-```erlang
-#{
-    <<"action">> => <<"deploy">>,
-    <<"task_id">> => TaskId,
-    <<"params">> => Params
-}
-```
-
-触发容器部署。部署过程中的流式日志不通过该 command response 返回，而是通过 `message` 的 `task_event` 上报。
-
-### start
-
-```erlang
-#{
-    <<"action">> => <<"start">>,
-    <<"target">> => Target
-}
-```
-
-### stop
-
-```erlang
-#{
-    <<"action">> => <<"stop">>,
-    <<"target">> => Target,
-    <<"timeout_seconds">> => TimeoutSeconds
-}
-```
-
-### kill
-
-```erlang
-#{
-    <<"action">> => <<"kill">>,
-    <<"target">> => Target,
-    <<"signal">> => Signal
-}
-```
-
-### remove
-
-```erlang
-#{
-    <<"action">> => <<"remove">>,
-    <<"target">> => Target,
-    <<"force">> => Force,
-    <<"remove_volumes">> => RemoveVolumes
-}
-```
-
-### config
-
-```erlang
-#{
-    <<"action">> => <<"config">>,
-    <<"target">> => Target,
-    <<"config">> => Config
-}
-```
-
-更新容器配置文件。
-
-### Target
-
-容器目标使用 map 表示：
-
-```erlang
-#{
-    <<"name">> => ContainerName,
-    <<"id">> => ContainerId
-}
-```
-
-`name` 和 `id` 至少一个非空；优先使用 `name`，`name` 为空时使用 `id`。
-
-## 异步消息
-
-`message` 不带 `Ref`，不要求对端回复。
-
-### efka -> iot: data
-
-```erlang
-{<<"message">>, {<<"data">>, #{
-    <<"route_key">> => RouteKey,
-    <<"metric">> => Metric
-}}}
-```
-
-用于 `efka` 上报业务指标数据。
-
-### efka -> iot: task_event
-
-```erlang
-{<<"message">>, {<<"task_event">>, #{
-    <<"task_id">> => TaskId,
-    <<"type">> => Type,
-    <<"stream">> => Stream
-}}}
-```
-
-任务事件流关闭时：
-
-```erlang
-{<<"message">>, {<<"task_event">>, #{
-    <<"task_id">> => TaskId,
-    <<"type">> => <<"close">>,
-    <<"stream">> => Reason
-}}}
-```
-
-`task_event` 本身不携带 `uuid`。`iot` 在接收该消息时使用当前已鉴权 `ssl_channel` 绑定的 host UUID，把事件路由到内部任务进程 `{UUID, TaskId}`。HTTP 页面通过 SSE 订阅时也必须使用同一组参数：
-
-```http
-GET /event_stream?uuid=<host_uuid>&task_id=<task_id>
-```
-
-`iot` 会为每个 `{UUID, TaskId}` 维护一个独立的任务进程，用于缓存最近的部署日志、支持多个 SSE listener，并在收到 close 事件后结束事件流。
-
-### efka -> iot: ping
-
-```erlang
-{<<"message">>, <<"ping">>}
-```
-
-用于 TLS 长连接的应用层保活。`efka` 在鉴权成功后周期发送，当前发送间隔为 30 秒。
-
-`iot` 收到后回复：
-
-```erlang
-{<<"message">>, <<"pong">>}
-```
-
-`ping/pong` 只表示 TLS 连接仍可读写，不参与 host online/offline 判定。host 上下线仍由 UDP 心跳和 `iot_host` 本地连接状态共同维护。
-
-### iot -> efka: pub
-
-```erlang
-{<<"message">>, {<<"pub">>, #{
-    <<"topic">> => Topic,
-    <<"qos">> => Qos,
-    <<"content">> => Content
-}}}
-```
-
-用于 `iot` 向 `efka` 本地订阅系统发布 topic 消息。
-
-## 状态与超时
-
-- `efka` 鉴权超时时间：5 秒。
-- `iot` command inflight 超时时间：60 秒。
-- `iot` SSL channel 空闲超时时间：120 秒。120 秒内没有收到任何 TLS 包，包括 `ping`、业务 `message`、`request`、`command_response`，`iot` 会主动关闭该连接。
-- `iot` 管理多个 `efka` 时，每个连接有独立 `ssl_channel` 和独立 inflight 表。
-- command 超时后，`iot` 删除 inflight 记录；之后如果迟到的 `command_response` 到达，会被视为未预期响应。
-
-## UDP 心跳
-
-`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。TLS control channel 和 UDP 心跳共用 `iot_server.host`，分别使用 `tls_port` 和 `udp_port`。UDP 心跳包使用 HMAC-SHA256 校验，HMAC key 为 `SHA256(auth.token)`。
-
-详细格式见 [heartbeat.md](heartbeat.md)。
-
-## 兼容性
-
-当前协议不兼容旧 tuple：
-
-- 旧容器管理：`{request, Ref, {container_request, ...}}`
-- 旧容器回复：`{response, Ref, {container_response, ...}}`
-- 旧授权控制：`{message, {auth_control, Command}}`
-- 已移除的 auth command：`{command, Ref, {auth, activate | deactivate}}`
-- 旧 Ref：Erlang `reference()`，例如 `make_ref()` 生成的值。
-
-如果需要滚动升级，应先增加临时兼容分支或引入协议版本协商。

docs/heartbeat.md

@@ -1,52 +0,0 @@
-# UDP 心跳
-
-`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。该心跳只表示主机存活，不依赖 TLS control channel 是否在线。
-
-## 配置
-
-TLS 和 UDP 共用同一个 `iot_server.host`：
-
-```erlang
-{iot_server, [
-  {host, "localhost"},
-  {tls_port, 443},
-  {udp_port, 18080}
-]},
-
-{heartbeat, [
-  {interval, 5000}
-]},
-
-{auth, [
-  {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"},
-  {token, "zpxlkvmqwnbghytrujsdieofazxcvbnm"}
-]}
-```
-
-- `tls_port` 用于 `efka_iot_client` 建立 TLS 连接。
-- `udp_port` 用于 `efka_iot_heartbeat` 发送 UDP 心跳。
-- `heartbeat.interval` 是发送间隔，单位毫秒，默认 5000。
-- UDP 心跳和 TLS 鉴权复用 `auth.uuid` 和 `auth.token`。
-
-## 包格式
-
-```erlang
-<<
-    Version:8,
-    UuidLen:16,
-    UUID:UuidLen/binary,
-    Timestamp:64/unsigned-big,
-    Nonce:16/binary,
-    Mac:32/binary
->>
-```
-
-`Mac` 使用 HMAC-SHA256：
-
-```erlang
-HeartbeatSecret = crypto:hash(sha256, Token),
-Payload = <<Version:8, UuidLen:16, UUID:UuidLen/binary, Timestamp:64/unsigned-big, Nonce:16/binary>>,
-Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload)
-```
-
-其中 `Token` 是 `auth.token`。`iot` 侧注册 efka client 时保存同样的 `SHA256(Token)` 派生值，并用它校验心跳。

docs/service_websocket_protocol.md

@@ -1,439 +0,0 @@
-# EFKA 与 Service WebSocket 交互协议
-
-本文档描述部署在 `efka` 主机上的 service 与 `efka` 之间的 WebSocket 通讯逻辑。当前接入实现位于：
-
-- [efka_service_channel.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service_channel.erl)
-- [service.proto](/usr/local/code/cloudkit/efka/apps/efka/proto/service.proto)
-- [efka_subscription.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_subscription.erl)
-- [efka_service.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service.erl)
-
-## 1. 连接入口
-
-`efka` 启动时会创建 Cowboy WebSocket server，监听配置来自 `efka.config` 的 `websocket_server`：
-
-```erlang
-{websocket_server, [
-    {port, 18080},
-    {acceptors, 10},
-    {max_connections, 1024},
-    {backlog, 1024}
-]}
-```
-
-WebSocket 路由固定为：
-
-```http
-GET /ws
-```
-
-连接建立后，每个 WebSocket 连接对应一个 `efka_service_channel` 进程。
-
-当前协议没有独立的认证字段。service 必须先发送 `register` request 完成注册；未注册前只能处理 `register`，其它 request 会返回 `invalid request`，cast 会被忽略。
-
-Cowboy WebSocket ping 帧由 `efka_service_channel` 直接回复 pong：
-
-```erlang
-websocket_handle(ping, State) -> {reply, pong, State}
-```
-
-## 2. 二进制帧格式
-
-service 与 `efka` 之间使用 WebSocket binary frame。每个业务 frame 的第 1 个字节是自定义帧类型，剩余字节是 protobuf payload。
-
-```text
-<frame_type:1 byte><protobuf_payload:N bytes>
-```
-
-帧类型：
-
-| frame_type | 十六进制 | 方向 | 语义 | protobuf 类型 |
-| --- | --- | --- | --- | --- |
-| `REQUEST` | `0x01` | service -> efka | 请求，需要 efka 回复 | `ServiceRequest` |
-| `REPLY` | `0x02` | efka -> service | request 的响应 | `ServiceReply` |
-| `CAST` | `0x03` | 双向 | 单向消息，不要求回复 | `ServiceCast` |
-
-`efka_service_channel` 当前只处理 service 发来的 `REQUEST` 和 `CAST`。收到未知格式时只记录日志，不主动发送错误响应。
-
-## 3. Protobuf 定义
-
-协议定义来自 `apps/efka/proto/service.proto`：
-
-```protobuf
-syntax = "proto3";
-
-message ServiceRequest {
-  uint32 packet_id = 1;
-
-  message Register {
-    string service_id = 1;
-  }
-
-  message Subscribe {
-    string topic = 1;
-  }
-
-  oneof request {
-    Register register = 10;
-    Subscribe subscribe = 11;
-  }
-}
-
-message ServiceReply {
-  uint32 packet_id = 1;
-
-  message Error {
-    int32 code = 1;
-    string message = 2;
-  }
-
-  oneof reply {
-    bytes result = 10;
-    Error error = 11;
-  }
-}
-
-message ServiceCast {
-  message MetricData {
-    bytes route_key = 1;
-    bytes metric = 2;
-  }
-
-  message TopicEvent {
-    string topic = 1;
-    bytes content = 2;
-  }
-
-  oneof body {
-    TopicEvent topic_event = 10;
-    MetricData metric_data = 11;
-  }
-}
-```
-
-注意：
-
-- `ServiceRequest.packet_id` 由 service 生成，用于匹配 request 和 reply。
-- `ServiceReply.packet_id` 必须等于原 request 的 `packet_id`。
-- `ServiceReply.result` 当前成功值一般为 `<<"ok">>`。
-- `ServiceReply.error.code` 当前失败时统一使用 `-1`。
-- `MetricData.route_key` 和 `MetricData.metric` 是 bytes。
-- `TopicEvent.topic` 是 string，`TopicEvent.content` 是 bytes。
-
-## 4. request/reply 语义
-
-### 4.1 register
-
-service 建立 WebSocket 后应先发送注册请求：
-
-```protobuf
-ServiceRequest {
-  packet_id: 1
-  register {
-    service_id: "service-a"
-  }
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x01 ++ protobuf(ServiceRequest)
-```
-
-`efka` 处理逻辑：
-
-1. `efka_service_channel` 解码 `ServiceRequest`。
-2. 调用 `efka_service_sup:start_service(ServiceId)` 启动或复用 service 进程。
-3. 调用 `efka_service:attach_channel(ServicePid, ChannelPid)` 绑定当前 WebSocket channel。
-4. 通过 `efka_service_model:insert/1` 写入 service 记录：
-
-```erlang
-#service{
-    service_id = ServiceId,
-    container_name = <<>>,
-    status = ?SERVICE_RUNNING,
-    meta_data = #{},
-    create_ts = efka_util:timestamp(),
-    update_ts = efka_util:timestamp()
-}
-```
-
-如果该 `service_id` 是新记录，`status` 会写入 `?SERVICE_RUNNING`。如果该 `service_id` 已存在，当前 `efka_service_model:insert/1` 只更新 `meta_data`、`container_name` 和 `update_ts`，不会覆盖已有 `status`。
-
-5. `efka_service_channel` 状态更新为：
-
-```erlang
-#state{
-    service_id = ServiceId,
-    service_pid = ServicePid,
-    is_registered = true
-}
-```
-
-成功响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 1
-  result: "ok"
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x02 ++ protobuf(ServiceReply)
-```
-
-失败响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 1
-  error {
-    code: -1
-    message: "attach channel failed"
-  }
-}
-```
-
-常见失败场景：
-
-- 同一个 `service_id` 已有存活 channel，`efka_service:attach_channel/2` 返回 `{error, <<"channel exists">>}`。
-- 当前实现对外统一返回 `"attach channel failed"`，详细原因只写日志。
-
-### 4.2 subscribe
-
-注册成功后，service 可以订阅 topic：
-
-```protobuf
-ServiceRequest {
-  packet_id: 2
-  subscribe {
-    topic: "device/*/event"
-  }
-}
-```
-
-`efka` 处理逻辑：
-
-1. 要求当前 channel 已注册，即 `is_registered = true`。
-2. 调用 `efka_subscription:subscribe(Topic, ChannelPid)`。
-3. 订阅成功后，把 topic 加入 `efka_service_channel` 的 `subscribed_topics` 集合。
-
-成功响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 2
-  result: "ok"
-}
-```
-
-失败响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 2
-  error {
-    code: -1
-    message: "invalid topic name"
-  }
-}
-```
-
-未注册时发送 subscribe 会走通用错误：
-
-```protobuf
-ServiceReply {
-  packet_id: 2
-  error {
-    code: -1
-    message: "invalid request"
-  }
-}
-```
-
-## 5. cast 语义
-
-### 5.1 service -> efka: metric_data
-
-service 通过 `CAST` 上报指标数据：
-
-```protobuf
-ServiceCast {
-  metric_data {
-    route_key: "device/a/temp"
-    metric: "{\"value\":23.5}"
-  }
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x03 ++ protobuf(ServiceCast)
-```
-
-`efka` 处理逻辑：
-
-1. `efka_service_channel` 解码 `ServiceCast`。
-2. 要求当前 channel 已注册。
-3. 调用：
-
-```erlang
-efka_service:metric_data(ServicePid, RouteKey, Metric)
-```
-
-4. `efka_service` 再调用：
-
-```erlang
-efka_iot_client:metric_data(RouteKey, Metric)
-```
-
-5. `efka_iot_client` 通过 efka 与 iot 的 TLS 长连接把数据上报给 iot：
-
-```erlang
-{<<"message">>, {<<"data">>, #{
-    <<"route_key">> => RouteKey,
-    <<"metric">> => Metric
-}}}
-```
-
-`metric_data` 是单向消息，service 不会收到 reply。未注册或未知 cast body 会被忽略。
-
-### 5.2 efka -> service: topic_event
-
-当 `iot` 通过 efka 与 iot 的 TLS channel 下发 pub 消息到 `efka` 时，`efka_iot_client` 会调用：
-
-```erlang
-efka_subscription:publish(Topic, Qos, Content)
-```
-
-`efka_subscription` 根据 topic 匹配已订阅的 `efka_service_channel`。匹配成功后向 channel 进程发送：
-
-```erlang
-{topic_broadcast, Topic, Content}
-```
-
-`efka_service_channel` 再推送 WebSocket `CAST` 给 service：
-
-```protobuf
-ServiceCast {
-  topic_event {
-    topic: "device/a/event"
-    content: "payload bytes"
-  }
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x03 ++ protobuf(ServiceCast)
-```
-
-`topic_event` 是单向消息，不要求 service 回复。
-
-## 6. topic 匹配规则
-
-`efka_subscription` 当前使用 `/` 拆分 topic：
-
-```erlang
-binary:split(Topic, <<$/>>, [global])
-```
-
-支持两种通配符：
-
-| 通配符 | 语义 |
-| --- | --- |
-| `*` | 单级匹配，只匹配一个 segment |
-| `+` | 多级匹配，只能出现在最后一段，且至少匹配一个剩余 segment |
-
-示例：
-
-| 订阅 topic | 发布 topic | 是否匹配 |
-| --- | --- | --- |
-| `device/a/temp` | `device/a/temp` | 是 |
-| `device/*/temp` | `device/a/temp` | 是 |
-| `device/*/temp` | `device/a/b/temp` | 否 |
-| `device/+` | `device/a` | 是 |
-| `device/+` | `device/a/temp` | 是 |
-| `device/+` | `device` | 否 |
-
-同一个 `efka_service_channel` 订阅多个 topic 时，同一次 publish 最多投递一次。
-
-## 7. QoS 与遗留消息
-
-`efka_subscription:publish(Topic, Qos, Content)` 中的 `Qos` 影响没有订阅者时的行为：
-
-- `Qos = 0`：没有匹配订阅者时直接丢弃。
-- `Qos /= 0`：没有匹配订阅者时暂存到 `remaining_messages`。
-
-当后续有 service 订阅能匹配这些 topic 时，`efka_subscription` 会把对应遗留消息推送给该 service，并从 `remaining_messages` 移除。
-
-当前 `remaining_messages` 存在内存中，不持久化；`efka` 重启后会丢失。
-
-## 8. 生命周期与清理
-
-### WebSocket channel 关闭
-
-`efka_service_channel:terminate/3` 会执行：
-
-1. 调用 `efka_subscription:unsubscribe_all(ChannelPid)` 清理该 channel 的所有订阅。
-2. 如果 channel 已完成 register，则调用：
-
-```erlang
-efka_service_model:change_status(ServiceId, ?SERVICE_STOPPED)
-```
-
-也就是把 service 状态更新为停止。
-
-### service 进程退出
-
-`efka_service_channel` monitor 绑定的 `efka_service` 进程。如果 service 进程退出，channel 会停止：
-
-```erlang
-websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State) ->
-    {stop, State#state{service_pid = undefined}}
-```
-
-`efka_service` 同时也 monitor channel。channel 退出时，`efka_service` 会清空自身的 `channel_pid`。
-
-## 9. 完整交互流程
-
-典型流程：
-
-```text
-service                         efka/efka_service_channel              efka_service              efka_subscription
-  |                                      |                              |                            |
-  |-- WebSocket connect /ws ------------>|                              |                            |
-  |                                      |                              |                            |
-  |-- REQUEST register ----------------->|                              |                            |
-  |                                      |-- start_service(service_id) ->|                            |
-  |                                      |-- attach_channel ----------->|                            |
-  |<------------- REPLY result="ok" -----|                              |                            |
-  |                                      |                              |                            |
-  |-- REQUEST subscribe ---------------->|                              |                            |
-  |                                      |---------------- subscribe ------------------------------>|
-  |<------------- REPLY result="ok" -----|                              |                            |
-  |                                      |                              |                            |
-  |-- CAST metric_data ----------------->|                              |                            |
-  |                                      |-- metric_data -------------->|                            |
-  |                                      |                              |-- efka_iot_client:metric_data -> iot
-  |                                      |                              |                            |
-  |<------------- CAST topic_event ------|<--------------- topic_broadcast -----------------------|
-  |                                      |                              |                            |
-  |-- WebSocket close ------------------>|                              |                            |
-  |                                      |-- unsubscribe_all -------------------------------------->|
-  |                                      |-- change_status(service_id, stopped)
-```
-
-## 10. 当前限制
-
-- 没有认证字段，service 只通过 `service_id` 注册。
-- `register` 可重复连接同一个 `service_id`，但同一时间只允许一个活跃 channel 绑定到 `efka_service`。
-- request 只支持 `register` 和 `subscribe`。
-- cast 只支持 `metric_data` 和 `topic_event`。
-- `subscribed_topics` 目前只在 `efka_service_channel` 内记录，关闭时实际清理由 `efka_subscription:unsubscribe_all/1` 完成。
-- `efka_subscription` 当前基于列表扫描匹配，订阅规模很大时需要后续优化为索引结构。

docs/todo.md

@@ -1,50 +0,0 @@
-# TODO
-
-## 协议层
-
-### 高优先级
-
-- 将 `message.proto` 收敛为 `efka` 与 `iot` 共用的一份单一事实来源，避免协议定义漂移和双端重复维护。
-
-### 中优先级
-
-- 增强 protobuf 映射和 Docker JSON 生成相关的自动化契约测试。
-
-## Docker 层
-
-### 高优先级
-
-- 将 `docker_commands` 拆分为更清晰的层次：Docker HTTP 客户端、Docker JSON 构造层、以及 efka 本地补丁逻辑。
-- 明确 `docker_task_reporter` 的投递语义；当前只有内存缓冲，还缺少持久化能力和队列上限控制。
-
-### 中优先级
-
-- 丰富 `docker_deploy_manager` 的任务跟踪状态，把部署元数据、耗时、阶段、失败原因等信息纳入统一管理，避免后续再回头重构。
-- 统一 `docker_container_service` 的返回结构，避免调用方分支处理 `ok | {ok, binary()} | {error, binary()}` 这种混合形式。
-- 消除 `docker_commands` 中重复的 Docker HTTP 响应解析逻辑。
-
-### 低优先级
-
-- 对于像 `docker_events.erl` 这样当前未启用的模块，要么清理掉，要么补充说明其保留原因。
-
-## 订阅与通道层
-
-### 高优先级
-
-- 将 `efka_subscription` 当前基于列表扫描的匹配方式替换为更可扩展的索引结构，例如 trie 或基于 ETS 的索引。
-
-### 中优先级
-
-- 继续收敛 `efka_iot_client` 的职责，将传输层状态管理与 request/cast 协议处理进一步拆开。
-- 评估并落实 `efka_service_channel` 中 `subscribed_topics` 集合的用途；如果只是被动保存状态，则应简化。
-
-### 低优先级
-
-- 如果 `efka_iot_client` 的缓存指标量继续增长，可以为缓存刷出增加批量发送或节流机制。
-
-## 基础设施层
-
-### 低优先级
-
-- 如果上传相关能力已经不再属于当前运行时，移除或补充说明残留的启动逻辑，例如 `ensure_upload_dir/0`。
-- 评估是否将 `efka_logger` 与 OTP `logger` 统一，避免长期维护两套日志链路。

env.file

@@ -1,15 +0,0 @@
-[dev]
-EFKA_DETS_DIR="/usr/local/var/lib/efka/dets/"
-EFKA_IOT_HOST="localhost"
-EFKA_AUTH_UUID="qbxmjyzrkpntfgswaevodhluicqzxplkm"
-EFKA_AUTH_TOKEN="zpxlkvmqwnbghytrujsdieofazxcvbnm"
-EFKA_DOCKER_ROOT_DIR="/usr/local/var/lib/efka/docker/"
-EFKA_MNESIA_DIR="/usr/local/var/lib/efka/mnesia"
-
-[prod]
-EFKA_DETS_DIR="/var/lib/efka/dets/"
-EFKA_IOT_HOST="localhost"
-EFKA_AUTH_UUID="qbxmjyzrkpntfgswaevodhluicqzxplkm"
-EFKA_AUTH_TOKEN="zpxlkvmqwnbghytrujsdieofazxcvbnm"
-EFKA_DOCKER_ROOT_DIR="/var/lib/efka/docker/"
-EFKA_MNESIA_DIR="/var/lib/efka/mnesia"

message_pb.proto

@@ -22,31 +22,20 @@ message Pub {
     bytes content = 2;
 }
 
-message Command {
-    string command_type = 1;
-    bytes command = 2;
-}
-
 ///// 服务器主动推送的消息
 
+message AsyncCallReply {
+    // 0: 表示失败，1: 成功
+    uint32 code = 1;
+    string result = 2;
+    string message = 3;
+}
+
 // 部署逻辑
-message RPCDeploy {
+message Deploy {
     uint32 task_id = 1;
-    // json
+    string service_id = 2;
-    string config = 2;
+    string tar_url = 3;
-}
-
-message RPCStartContainer {
-    string container_name = 1;
-}
-
-message RPCStopContainer {
-    string container_name = 1;
-}
-
-message RPCConfigContainer {
-    string container_name = 1;
-    bytes config = 2;
 }
 
 // 获取task的logs
@@ -54,11 +43,18 @@ message FetchTaskLog {
     uint32 task_id = 1;
 }
 
+// 需要响应, 云端主动发起的调用; 提供给用户
+message Invoke {
+    string service_id = 1;
+    bytes payload = 2;
+    uint32 timeout = 3;
+}
+
 // 参数配置
-message ContainerConfig {
+message PushServiceConfig {
-    string container_name = 1;
+    string service_id = 1;
-    // 任意的数据格式
+    string config_json = 2;
-    bytes config = 2;
+    uint32 timeout = 3;
 }
 
 /////// EFKA主动上报的消息类型
@@ -72,12 +68,6 @@ message Data {
     bytes metric = 4;
 }
 
-message Event {
-    string service_id = 1;
-    uint32 event_type = 2;
-    string params = 3;
-}
-
 //#{<<"adcode">> => 0,<<"boot_time">> => 18256077,<<"city">> => <<>>,
 // <<"cpu_core">> => 2,
 // <<"cpu_load">> => 0,<<"cpu_temperature">> => 43.75,
@@ -109,3 +99,23 @@ message Ping {
     // 接口信息的定义: 每个接口的信息, 采用json格式传输，没有办法提前定义
     string interfaces = 13;
 }
+
+// Inform消息
+message ServiceInform {
+    string service_id = 1;
+    string props = 2;
+    uint32 status = 3;
+    uint32 timestamp = 4;
+}
+
+message Event {
+    string service_id = 1;
+    uint32 event_type = 2;
+    string params = 3;
+}
+
+// 告警信息
+message Alarm {
+    string service_id = 1;
+    string params = 2;
+}

rebar.config

@@ -1,15 +1,12 @@
 {erl_opts, [debug_info]}.
-
-{project_app_dirs, ["apps/*"]}.
-
-{plugins, [
-    {rebar3_gpb_plugin, ".*", {git, "https://github.com/lrascao/rebar3_gpb_plugin.git", {tag, "2.23.8"}}}
-]}.
-
 {deps, [
     {sync, ".*", {git, "https://github.com/rustyio/sync.git", {branch, "master"}}},
+    {jiffy, ".*", {git, "https://github.com/davisp/jiffy.git", {tag, "1.1.2"}}},
+    {gpb, ".*", {git, "https://github.com/tomas-abrahamsson/gpb.git", {tag, "4.20.0"}}},
+    {jiffy, ".*", {git, "https://github.com/davisp/jiffy.git", {tag, "1.1.1"}}},
     {cowboy, ".*", {git, "https://github.com/ninenines/cowboy.git", {tag, "2.10.0"}}},
-    {gun, ".*", {git, "https://github.com/ninenines/gun.git", {tag, "2.2.0"}}}
+    {parse_trans, ".*", {git, "https://github.com/uwiger/parse_trans", {tag, "3.0.0"}}},
+    {lager, ".*", {git,"https://github.com/erlang-lager/lager.git", {tag, "3.9.2"}}}
 ]}.
 
 {relx, [{release, {efka, "0.1.0"},
@@ -17,10 +14,18 @@
           sasl]},
 
         {mode, dev},
-        {include_erts, false},
 
-        {sys_config_src, "./config/sys.config.src"},
+        %% automatically picked up if the files
-        {vm_args_src, "./config/vm.args.src"}
+        %% exist but can be set manually, which
+        %% is required if the names aren't exactly
+        %% sys.config and vm.args
+        {sys_config, "./config/sys.config"},
+        {vm_args, "./config/vm.args"}
+
+        %% the .src form of the configuration files do
+        %% not require setting RELX_REPLACE_OS_VARS
+        %% {sys_config_src, "./config/sys.config.src"},
+        %% {vm_args_src, "./config/vm.args.src"}
 ]}.
 
 {profiles, [{prod, [{relx,
@@ -34,4 +39,11 @@
                      ]
             }]}]}.
 
+{project_plugins, [
+    %% 或从 Git 仓库拉取最新版本
+    {pc, {git, "https://github.com/blt/port_compiler.git", {tag, "v1.15.0"}}}
+]}.
+
+{erl_opts, [{parse_transform,lager_transform}]}.
+
 {rebar_packages_cdn, "https://hexpm.upyun.com"}.

run

@@ -1,10 +1,6 @@
 #! /bin/sh
 
-export EFKA_DETS_DIR="/usr/local/var/lib/efka/dets/"
+rebar3 compile
-export EFKA_IOT_HOST="localhost"
+rebar3 release
-export EFKA_AUTH_UUID="qbxmjyzrkpntfgswaevodhluicqzxplkm"
-export EFKA_AUTH_TOKEN="zpxlkvmqwnbghytrujsdieofazxcvbnm"
-export EFKA_DOCKER_ROOT_DIR="/usr/local/var/lib/efka/docker/"
-export EFKA_MNESIA_DIR="/usr/local/var/lib/efka/mnesia"
 
 _build/default/rel/efka/bin/efka console