add debug

CODE_LOGIC_OVERVIEW.md

@@ -1,288 +0,0 @@
-# EFKA 当前代码逻辑梳理
-
-> 本文档记录当前阅读代码后对项目逻辑的理解，主要基于 `apps/efka/src/`、`apps/efka/include/`、`config/` 和 protobuf 生成文件。尚未覆盖每个边界条件和全部测试行为。
-
-## 1. 项目定位
-
-这个项目是一个 Erlang/OTP 应用，核心作用像一个边缘侧 agent：
-
-- 对本机微服务开放 WebSocket 接入，微服务注册、订阅 topic、上报指标。
-- 与上游服务器建立 TLS 长连接，完成鉴权、上报数据、接收远程命令。
-- 通过 Docker API 管理本机容器，包括部署、启动、停止、杀掉、删除、更新配置和列表查询。
-- 用 DETS 做少量本地持久化，包括服务状态和离线缓存包。
-
-依赖主要有：
-
-- `cowboy`：本地 WebSocket server。
-- `gun`：通过 Unix Socket 访问 Docker HTTP API。
-- `ssl`：连接上游 TLS server。
-- OTP `json` 模块：JSON 编解码。
-- `gpb`：protobuf 代码生成。
-
-## 2. 启动流程
-
-应用入口是 `efka_app:start/2`。
-
-启动步骤：
-
-1. 设置 Erlang IO 编码为 unicode。
-2. 设置 `fullsweep_after`，试图加快进程内存回收。
-3. 启动 Cowboy WebSocket server。
-4. 启动顶层 supervisor `efka_sup`。
-
-WebSocket server：
-
-- 从 `config/sys.config` 读取 `efka.websocket_server`。
-- 监听 `/ws`。
-- 请求处理模块是 `efka_service_channel`。
-
-`efka_sup` 启动的子进程：
-
-- `efka_logger`：部署日志落盘。
-- `efka_service_sup`：动态管理每个已注册微服务对应的 `efka_service` 进程。
-- `efka_service_model`：DETS 服务状态表。
-- `efka_subscription`：本地 topic 订阅中心。
-- `efka_iot_client`：连接上游 TLS server 的状态机。
-
-`docker` application 启动的子进程：
-
-- `docker_task_reporter`：部署任务事件缓冲和转发。
-- `docker_deploy_manager`：部署任务管理器。
-
-`docker_events` 代码存在于 `apps/docker`，但目前默认不会启动。
-
-## 3. 两条通信链路
-
-项目里有两条主要通信链路。
-
-### 3.1 微服务到 EFKA：WebSocket
-
-本机微服务连接 `ws://<host>:18080/ws`。
-
-协议处理在 `efka_service_channel`：
-
-- WebSocket 收到 `ping` 直接回复 `pong`。
-- 收到 binary 帧时，第一个字节是帧类型。
-- `FRAME_REQUEST = 0x01`，用 `efka_service_pb` 解码为 `ServiceRequest`。
-- `FRAME_CAST = 0x03`，用 `efka_service_pb` 解码为 `ServiceCast`。
-- 回复时使用 `FRAME_REPLY = 0x02`。
-
-支持的微服务请求：
-
-- `register`：注册服务。
-- `subscribe`：订阅 topic。
-
-支持的微服务 cast：
-
-- `metric_data`：上报指标数据。
-
-注册流程：
-
-1. 微服务发送 `ServiceRequest.Register`，包含 `service_id`。
-2. `efka_service_channel` 调用 `efka_service_sup:start_service(ServiceId)`。
-3. 如果该服务进程不存在，则动态启动一个 `efka_service`；如果已存在，复用已有 pid。
-4. `efka_service_channel` 调用 `efka_service:attach_channel(ServicePid, self())` 绑定当前 WebSocket channel。
-5. `efka_service` 只允许绑定一个存活 channel；已有 channel 存活时返回 `channel exists`。
-6. 注册成功后写入 `efka_service_model`，服务状态设为 running。
-7. channel 退出时，取消全部订阅，并把服务状态改为 stopped。
-
-指标上报流程：
-
-1. 微服务发送 `ServiceCast.MetricData`。
-2. `efka_service_channel` 调用 `efka_service:metric_data(ServicePid, RouteKey, Metric)`。
-3. `efka_service` 转发给 `efka_iot_client:metric_data(RouteKey, Metric)`。
-4. `efka_iot_client` 如果处于 activated 状态，直接发给上游；否则写入持久化 outbox。
-
-### 3.2 EFKA 到上游：TLS 长连接
-
-上游连接逻辑在 `efka_iot_client`，它是一个 `gen_statem`。
-
-状态包括：
-
-- `disconnected`：未连接。
-- `auth`：已连接，等待鉴权响应。
-- `restricted`：鉴权受限，不能正常推送数据，但可以接收部分授权命令。
-- `activated`：已激活，可正常收发。
-
-连接流程：
-
-1. 初始化后立即触发 `create_transport`。
-2. 读取 `efka.tls_server_address`，用 `ssl:connect/4` 建立 TLS 连接。
-3. TLS socket 使用 `{packet, 4}` 和 `{active, true}`。
-4. 连接成功后发送 `AuthRequest`。
-5. 收到鉴权成功 reply 后进入 `activated`，并触发 outbox 刷出。
-6. 连接或鉴权失败时关闭 socket，5 秒后重连。
-
-上游协议同样用第一个字节区分帧类型：
-
-- `FRAME_REQUEST = 0x01`
-- `FRAME_REPLY = 0x02`
-- `FRAME_CAST = 0x03`
-
-但 protobuf 使用的是 `message_pb`，不是微服务侧的 `efka_service_pb`。
-
-`efka_iot_client` 上报的内容：
-
-- `metric_data`：业务指标数据，activated 时实时发送，否则进入持久化 outbox。
-- `task_event_stream`：Docker 部署任务流式日志，只在 activated 时发送。
-- `close_task_event_stream`：任务结束事件，只在 activated 时发送。
-
-`efka_iot_client` 接收的内容：
-
-- `RequestFrame.container_request`：远程容器管理请求，交给 `docker_container_service`。
-- `CastFrame.command`：授权命令，`COMMAND_AUTH` 用于切换鉴权/受限状态。
-- `CastFrame.pub`：上游发布的 topic 消息，交给 `efka_subscription:publish/3`。
-
-## 4. 本地 Pub/Sub 逻辑
-
-`efka_subscription` 是本地订阅中心。
-
-订阅：
-
-- `efka_service_channel` 收到微服务 `subscribe` 后调用 `efka_subscription:subscribe(Topic, self())`。
-- 同一个 pid 对同一个 topic 重复订阅会被忽略。
-- 首次看到某个 subscriber pid 时，会 monitor 该 pid；pid 退出后自动移除订阅。
-
-topic 匹配规则：
-
-- `/` 分隔 topic components。
-- `*` 表示单级匹配。
-- `+` 表示多级匹配，但只能出现在末尾。
-- 完全匹配优先级最高，不过当前代码里的 `order` 字段只是计算并保存，没有实际排序使用。
-
-发布：
-
-- 上游发来的 `CastFrame.Pub` 会调用 `efka_subscription:publish(Topic, Qos, Content)`。
-- 如果匹配到订阅者，向对应 `efka_service_channel` 发送 `{topic_broadcast, Topic, Content}`。
-- `efka_service_channel` 再编码为 `ServiceCast.TopicEvent` 推送给微服务。
-- 如果没有订阅者且 `Qos != 0`，消息会暂存在 `remaining_messages`。
-- 新订阅建立时，会尝试把匹配的 remaining messages 补发给该订阅者。
-
-## 5. Docker 容器管理链路
-
-上游发来的容器请求最终进入 `docker_container_service:handle_request/1`。
-
-支持动作：
-
-- `list`：列出容器。
-- `deploy`：部署容器。
-- `start`：启动容器。
-- `stop`：停止容器。
-- `kill`：发送 kill 信号。
-- `remove`：删除容器。
-- `config`：更新容器配置文件。
-
-简单操作：
-
-- `start/stop/kill/remove/list` 直接调用 `docker_commands`。
-- `docker_commands` 通过短生命周期普通 `docker_client` 进程访问 `/var/run/docker.sock`。
-- HTTP 客户端用 `gun:open_unix/2`。
-- 流式请求使用 `{docker_client, Ref, Event}` 消息返回，事件包括 `{response, Status, Headers}`、`{data, Bin}`、`done` 和 `{error, Reason}`。
-
-部署操作：
-
-1. `docker_container_service` 收到 `deploy` 后调用 `docker_deploy_manager:deploy(TaskId, Params)`。
-2. `docker_deploy_manager` 根据 `docker.root_dir` 和参数里的 `container_name/container_dir` 确保容器目录存在。
-3. 它启动一个独立 `docker_deployer` 进程，并 monitor 该部署进程。
-4. `docker_deployer` 执行实际部署步骤。
-5. 部署过程事件写入 `docker_task_reporter`。
-6. `docker_task_reporter` 在上游连接 activated 时把事件转给 `efka_iot_client`；未激活时保留队列并定时重试。
-
-`docker_deployer` 当前部署步骤：
-
-1. 上报“开始部署容器”。
-2. 调用 `ensure_container_absent/2`，但当前实现只是上报“开始创建容器”，没有真正删除旧容器。
-3. 规范化镜像名，没有 tag 时补 `:latest`。
-4. 调用 Docker API 拉取镜像，`docker_deployer` 按 `Ref` 接收 `docker_client` 流式消息并上报拉取过程。
-5. 创建容器。
-6. 创建空的 `service.conf` 配置文件。
-7. 写部署摘要到 `efka_logger`。
-8. 关闭任务事件流，状态为 `success` 或 `fail`。
-
-创建容器参数：
-
-- `docker_container_builder` 把 protobuf 的 `DockerCreateOptions` 转成 Docker API JSON。
-- 会自动给容器注入环境变量 `CONTAINER_NAME=<name>`。
-- 会自动添加容器内配置路径 `/usr/local/etc/service.conf`。
-- 会自动把宿主机 `service.conf` bind 到容器内 `/usr/local/etc/service.conf`。
-
-配置更新：
-
-- `docker_container_service:update_container_config/2` 根据容器名找到目录。
-- 写入该目录下的 `service.conf`。
-- 容器目录映射由 `docker_helper` 通过 `.container_dir` 指针文件维护。
-
-## 6. 本地持久化
-
-### 6.1 `efka_service_model`
-
-使用 DETS 表 `service`。
-
-文件位置来自 `efka.dets_dir`，默认配置是：
-
-```erlang
-"/usr/local/code/tmp/dets/service.dets"
-```
-
-记录结构在 `apps/efka/include/efka_tables.hrl`：
-
-- `service_id`
-- `container_name`
-- `meta_data`
-- `status`
-- `create_ts`
-- `update_ts`
-
-用途：
-
-- 注册成功时插入或更新服务。
-- channel 关闭时把服务状态改成 stopped。
-- 支持查询所有服务、运行中服务和单个服务状态。
-
-### 6.2 `efka_iot_outbox`
-
-使用 append-only log 文件和 metadata 文件。
-
-作用：
-
-- 当 `efka_iot_client` 不在 activated 状态时，把待上报 packet 缓存下来。
-- `efka_iot_client` 激活后循环 `next -> send -> ack` 刷出。
-- 所有记录 ack 后会截断 log，下一轮从 seq 1 重新开始。
-
-## 7. 日志
-
-有两套日志：
-
-- OTP logger：在 `config/sys.config` 里配置到 console 和 `log/debug.log`。
-- `efka_logger`：自定义部署日志，按日期写到 `code:root_dir() ++ "/log/"` 下。
-
-`efka_logger` 主要用于部署任务摘要和失败原因。
-
-## 8. 配置项
-
-关键配置来自 `config/sys.config`：
-
-- `docker.root_dir`：容器相关目录根路径。
-- `dets_dir`：DETS 文件目录。
-- `websocket_server`：本地 WebSocket 监听配置。
-- `tls_server_address`：上游 TLS server 地址。
-- `auth`：上游鉴权信息。
-
-注意：
-
-- `efka_service_model` 打开 DETS 前假设 `dets_dir` 已存在，代码里没有显式创建目录。
-- `docker_client` 固定使用 `/var/run/docker.sock`，每次请求内部由短生命周期普通进程执行 open/request/close。
-
-## 9. 当前看到的几个注意点
-
-- `README.md` 描述的是 JSON-RPC 风格 WebSocket，但当前代码实际处理的是 binary protobuf 帧；README 可能已经过期。
-- `efka_service_channel` 里 `register` 只使用 `service_id`，没有使用 README 中提到的 `meta_data/container_name`。
-- `efka_iot_client:send_result_reply/3` 和 `send_error_reply/3` 编码 `ReplyFrame` 后没有加 `FRAME_REPLY` 前缀；接收侧是否期望裸 protobuf 需要确认。
-- `docker_deployer:ensure_container_absent/2` 当前没有真正确保旧容器不存在，只是上报日志。
-- `efka_subscription` 计算了 topic `order`，但匹配广播时没有使用优先级排序。
-- `docker_commands` 部分错误响应解码没有统一使用 `[return_maps]`，有些分支可能匹配不到 map。
-- `docker_events` 存在但未启动，且使用 shell 命令 `docker events`，与其他 Docker API 访问方式不同。
-
-## 10. 一句话主流程
-
-微服务通过 WebSocket 注册到 EFKA，本地 channel 把指标交给对应 `efka_service`，再由 `efka_iot_client` 通过 TLS 上报给上游；上游通过同一条 TLS 连接下发 pub/sub 消息和容器管理请求，pub/sub 再广播回本地微服务，容器请求则通过 Docker Unix Socket 在本机执行。

apps/docker/src/docker.app.src

@@ -1,15 +0,0 @@
-{application, docker,
- [{description, "Docker integration application"},
-  {vsn, "0.1.0"},
-  {registered, []},
-  {mod, {docker_app, []}},
-  {applications,
-   [
-    gun,
-    kernel,
-    stdlib
-   ]},
-  {modules, []},
-  {licenses, ["Apache-2.0"]},
-  {links, []}
- ]}.

apps/docker/src/docker_app.erl

@@ -1,18 +0,0 @@
-%%%-------------------------------------------------------------------
-%% @doc Docker application entry point.
-%% @end
-%%%-------------------------------------------------------------------
-
--module(docker_app).
-
--behaviour(application).
-
--export([start/2, stop/1]).
-
--spec start(term(), term()) -> {ok, pid()} | {error, term()}.
-start(_StartType, _StartArgs) ->
-    docker_sup:start_link().
-
--spec stop(term()) -> ok.
-stop(_State) ->
-    ok.

apps/docker/src/docker_client.erl

@@ -1,292 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc
-%%% Short-lived Docker API client process.
-%%%
-%%% Each request owns one process and one gun Unix socket connection.
-%%% The process exits after the Docker API response finishes.
-%%% @end
-%%%-------------------------------------------------------------------
--module(docker_client).
-
-%% API
--export([request/4, start_stream/5, start_stream_no_timeout/5]).
-
--define(DOCKER_SOCKET, "/var/run/docker.sock").
--define(RESPONSE_TIMEOUT, 5000).
--define(BODY_TIMEOUT, 10000).
--define(STREAM_BODY_TIMEOUT, 30000).
--define(CLIENT_TIMEOUT, 60000).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec request(Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
-    {ok, StatusCode :: integer(), RespHeaders :: proplists:proplist(), RespBody :: binary()} | {error, any()}.
-request(Method, Path, Body, Headers) when is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
-    Owner = self(),
-    Ref = make_ref(),
-    {Pid, MRef} = spawn_monitor(fun() -> request_worker(Owner, Ref, Method, Path, Body, Headers) end),
-    receive
-        {docker_client, Ref, {result, Result}} ->
-            erlang:demonitor(MRef, [flush]),
-            Result;
-        {'DOWN', MRef, process, Pid, Reason} ->
-            {error, {client_down, Reason}}
-    after ?CLIENT_TIMEOUT ->
-        exit(Pid, shutdown),
-        erlang:demonitor(MRef, [flush]),
-        {error, timeout}
-    end.
-
--spec start_stream(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
-    {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}.
-start_stream(Owner, Method, Path, Body, Headers)
-  when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
-    Ref = make_ref(),
-    {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, ?RESPONSE_TIMEOUT, ?STREAM_BODY_TIMEOUT) end),
-    {ok, Ref, Pid, MRef}.
-
--spec start_stream_no_timeout(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) ->
-    {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}.
-start_stream_no_timeout(Owner, Method, Path, Body, Headers)
-  when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
-    Ref = make_ref(),
-    {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, infinity, infinity) end),
-    {ok, Ref, Pid, MRef}.
-
-%%%===================================================================
-%%% Worker functions
-%%%===================================================================
-
--spec request_worker(pid(), reference(), string(), string(), binary(), list()) -> ok.
-request_worker(Owner, Ref, Method, Path, Body, Headers) ->
-    Result = do_request(Method, Path, Body, Headers),
-    send_owner(Owner, Ref, {result, Result}).
-
--spec stream_worker(pid(), reference(), string(), string(), binary(), list(), timeout(), timeout()) -> ok.
-stream_worker(Owner, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) ->
-    OwnerRef = erlang:monitor(process, Owner),
-    _Result = do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout),
-    erlang:demonitor(OwnerRef, [flush]),
-    ok.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec do_request(string(), string(), binary(), list()) ->
-    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
-do_request(Method, Path, Body, Headers) ->
-    case open_connection() of
-        {ok, ConnPid} ->
-            try
-                StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
-                receive_response(ConnPid, StreamRef)
-            after
-                close_connection(ConnPid)
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec do_stream(pid(), reference(), reference(), string(), string(), binary(), list(), timeout(), timeout()) ->
-    ok | {error, binary()}.
-do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) ->
-    case open_connection() of
-        {ok, ConnPid} ->
-            try
-                StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
-                receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout)
-            after
-                close_connection(ConnPid)
-            end;
-        {error, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason}
-    end.
-
--spec receive_response(pid(), reference()) ->
-    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
-receive_response(ConnPid, StreamRef) ->
-    receive
-        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} ->
-            receive_body(ConnPid, StreamRef, Status, Headers, <<>>);
-        {gun_response, ConnPid, StreamRef, fin, Status, Headers} ->
-            {ok, Status, Headers, <<>>};
-        {gun_error, ConnPid, StreamRef, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_error, ConnPid, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_down, ConnPid, _, Reason, _} ->
-            {error, {http_closed, Reason}}
-    after ?RESPONSE_TIMEOUT ->
-        {error, timeout}
-    end.
-
--spec receive_body(pid(), reference(), integer(), proplists:proplist(), iodata()) ->
-    {ok, integer(), proplists:proplist(), binary()} | {error, any()}.
-receive_body(ConnPid, StreamRef, Status, Headers, Acc) ->
-    receive
-        {gun_data, ConnPid, StreamRef, fin, Data} ->
-            Body = iolist_to_binary([Acc, Data]),
-            {ok, Status, Headers, Body};
-        {gun_data, ConnPid, StreamRef, nofin, Data} ->
-            receive_body(ConnPid, StreamRef, Status, Headers, [Acc, Data]);
-        {gun_error, ConnPid, StreamRef, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_error, ConnPid, Reason} ->
-            {error, {http_error, Reason}};
-        {gun_down, ConnPid, _, Reason, _} ->
-            {error, {http_closed, Reason}}
-    after ?BODY_TIMEOUT ->
-        {error, timeout}
-    end.
-
--spec receive_stream_response(pid(), reference(), reference(), pid(), reference(), timeout(), timeout()) -> ok | {error, binary()}.
-receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout) ->
-    receive
-        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} when Status >= 200, Status < 300 ->
-            send_owner(Owner, Ref, {response, Status, Headers}),
-            receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout);
-        {gun_response, ConnPid, StreamRef, fin, Status, Headers} when Status >= 200, Status < 300 ->
-            send_owner(Owner, Ref, {response, Status, Headers}),
-            send_owner(Owner, Ref, done),
-            ok;
-        {gun_response, ConnPid, StreamRef, nofin, Status, _Headers} ->
-            receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, <<>>, stream_error_body_timeout(BodyTimeout));
-        {gun_response, ConnPid, StreamRef, fin, Status, _Headers} ->
-            Reason = http_status_error(Status, <<>>),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, StreamRef, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_down, ConnPid, _, Reason0, _} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
-            {error, format_error({owner_down, Reason0})}
-    after ResponseTimeout ->
-        send_owner(Owner, Ref, {error, <<"处理超时"/utf8>>}),
-        {error, <<"timeout">>}
-    end.
-
--spec receive_stream_body(pid(), reference(), reference(), pid(), reference(), timeout()) -> ok | {error, binary()}.
-receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout) ->
-    receive
-        {gun_data, ConnPid, StreamRef, fin, Data} ->
-            maybe_send_stream_data(Owner, Ref, Data),
-            send_owner(Owner, Ref, done),
-            ok;
-        {gun_data, ConnPid, StreamRef, nofin, Data} ->
-            maybe_send_stream_data(Owner, Ref, Data),
-            receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout);
-        {gun_error, ConnPid, StreamRef, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_down, ConnPid, _, Reason0, _} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
-            {error, format_error({owner_down, Reason0})}
-    after BodyTimeout ->
-        send_owner(Owner, Ref, {error, <<"timeout">>}),
-        {error, <<"timeout">>}
-    end.
-
--spec receive_stream_error_body(pid(), reference(), reference(), pid(), reference(), integer(), iodata(), timeout()) ->
-    {error, binary()}.
-receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, Acc, BodyTimeout) ->
-    receive
-        {gun_data, ConnPid, StreamRef, fin, Data} ->
-            Reason = http_status_error(Status, iolist_to_binary([Acc, Data])),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_data, ConnPid, StreamRef, nofin, Data} ->
-            receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, [Acc, Data], BodyTimeout);
-        {gun_error, ConnPid, StreamRef, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_error, ConnPid, Reason0} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {gun_down, ConnPid, _, Reason0, _} ->
-            Reason = format_error(Reason0),
-            send_owner(Owner, Ref, {error, Reason}),
-            {error, Reason};
-        {'DOWN', OwnerRef, process, _Pid, Reason0} ->
-            {error, format_error({owner_down, Reason0})}
-    after BodyTimeout ->
-        Reason = http_status_error(Status, iolist_to_binary(Acc)),
-        send_owner(Owner, Ref, {error, Reason}),
-        {error, Reason}
-    end.
-
--spec stream_error_body_timeout(timeout()) -> timeout().
-stream_error_body_timeout(infinity) ->
-    infinity;
-stream_error_body_timeout(_BodyTimeout) ->
-    ?BODY_TIMEOUT.
-
--spec open_connection() -> {ok, pid()} | {error, any()}.
-open_connection() ->
-    case gun:open_unix(?DOCKER_SOCKET, #{}) of
-        {ok, ConnPid} ->
-            case gun:await_up(ConnPid) of
-                {ok, _} ->
-                    {ok, ConnPid};
-                {error, Reason} ->
-                    close_connection(ConnPid),
-                    {error, Reason}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec close_connection(pid()) -> ok.
-close_connection(ConnPid) ->
-    catch gun:close(ConnPid),
-    ok.
-
--spec send_owner(pid(), reference(), term()) -> ok.
-send_owner(Owner, Ref, Event) ->
-    Owner ! {docker_client, Ref, Event},
-    ok.
-
--spec maybe_send_stream_data(pid(), reference(), iodata()) -> ok.
-maybe_send_stream_data(Owner, Ref, Data) ->
-    case iolist_to_binary(Data) of
-        <<>> ->
-            ok;
-        DataBin ->
-            send_owner(Owner, Ref, {data, DataBin})
-    end.
-
--spec http_status_error(integer(), binary()) -> binary().
-http_status_error(Status, <<>>) ->
-    iolist_to_binary(io_lib:format("docker http status ~B", [Status]));
-http_status_error(Status, Body) when is_binary(Body) ->
-    StatusBin = integer_to_binary(Status),
-    <<"docker http status ", StatusBin/binary, ": ", Body/binary>>.
-
--spec format_error(term()) -> binary().
-format_error(Reason) when is_binary(Reason) ->
-    Reason;
-format_error(Reason) ->
-    iolist_to_binary(io_lib:format("~p", [Reason])).

apps/docker/src/docker_commands.erl

@@ -1,295 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 15. 9月 2025 16:11
-%%%-------------------------------------------------------------------
--module(docker_commands).
--author("anlicheng").
-
-%% API
--export([pull_image/1, check_image_exist/1]).
--export([create_container/2, check_container_exist/1, is_container_running/1,
-    start_container/1, stop_container/1, stop_container/2, remove_container/1, remove_container/3, kill_container/1, kill_container/2,
-    get_containers/0]).
-
--spec pull_image(Image :: binary()) -> {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()} | {error, Reason :: any()}.
-pull_image(Image) when is_binary(Image) ->
-    Url = lists:flatten(io_lib:format("/images/create?fromImage=~s", [binary_to_list(Image)])),
-    docker_client:start_stream_no_timeout(self(), "POST", Url, <<>>, []).
-
--spec check_image_exist(Image :: binary()) -> boolean().
-check_image_exist(Image) when is_binary(Image) ->
-    EncodedImage = uri_string:quote(Image),
-    Url = lists:flatten(io_lib:format("/images/~s/json", [binary_to_list(EncodedImage)])),
-    case docker_client:request("GET", Url, <<"">>, []) of
-        {ok, 200, _Headers, _Resp} ->
-            true;
-        {ok, 404, _, _} ->
-            false;
-        _ ->
-            false
-    end.
-
--spec create_container(ContainerDir :: string(), Params :: map()) ->
-    {ok, ContainerId :: binary()} | {error, Reason :: any()}.
-create_container(ContainerDir, #{<<"container_name">> := ContainerName0, <<"create">> := CreateOpts})
-    when is_list(ContainerDir), is_binary(ContainerName0), is_map(CreateOpts) ->
-    Url = lists:flatten(io_lib:format("/containers/create?name=~s", [binary_to_list(ContainerName0)])),
-    Options = docker_container_builder:build_options(ContainerName0, ContainerDir, CreateOpts),
-    display_options(Options),
-
-    Body = iolist_to_binary(json:encode(Options)),
-
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-
-    case docker_client:request("POST", Url, Body, Headers) of
-        {ok, 201, _Headers, Resp} ->
-            case catch json:decode(Resp) of
-                #{<<"Id">> := ContainerId} when is_binary(ContainerId) ->
-                    {ok, ContainerId};
-                _ ->
-                    {error, Resp}
-            end;
-        {ok, _StatusCode, _, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end;
-create_container(_ContainerDir, _Params) ->
-    {error, <<"invalid container params">>}.
-
--spec is_container_running(ContainerId :: binary()) -> boolean().
-is_container_running(ContainerId) when is_binary(ContainerId) ->
-    case inspect_container(ContainerId) of
-        {ok, #{<<"State">> := #{<<"Running">> := Running}}} ->
-            Running;
-        {error, _} ->
-            false
-    end.
-
--spec check_container_exist(ContainerName :: binary()) -> boolean().
-check_container_exist(ContainerName) when is_binary(ContainerName) ->
-    case inspect_container(ContainerName) of
-        {ok, #{<<"Id">> := Id}} when is_binary(Id) ->
-            true;
-        _ ->
-            false
-    end.
-
--spec start_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-start_container(ContainerName) when is_binary(ContainerName) ->
-    Url = lists:flatten(io_lib:format("/containers/~s/start", [binary_to_list(ContainerName)])),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("POST", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, 304, _Headers, _} ->
-            {error, <<"container already started">>};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec stop_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-stop_container(ContainerName) when is_binary(ContainerName) ->
-    stop_container(ContainerName, 0).
-
--spec stop_container(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> ok | {error, Reason :: binary()}.
-stop_container(ContainerName, TimeoutSeconds) when is_binary(ContainerName), is_integer(TimeoutSeconds), TimeoutSeconds >= 0 ->
-    Url = build_stop_container_url(ContainerName, TimeoutSeconds),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("POST", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, 304, _Headers, _} ->
-            {error, <<"container already stopped">>};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec kill_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-kill_container(ContainerName) when is_binary(ContainerName) ->
-    kill_container(ContainerName, <<>>).
-
--spec kill_container(ContainerName :: binary(), Signal :: binary()) -> ok | {error, Reason :: binary()}.
-kill_container(ContainerName, Signal) when is_binary(ContainerName), is_binary(Signal) ->
-    Url = build_kill_container_url(ContainerName, Signal),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("POST", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec remove_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
-remove_container(ContainerName) when is_binary(ContainerName) ->
-    remove_container(ContainerName, false, false).
-
--spec remove_container(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> ok | {error, Reason :: binary()}.
-remove_container(ContainerName, Force, RemoveVolumes)
-  when is_binary(ContainerName), is_boolean(Force), is_boolean(RemoveVolumes) ->
-    Url = build_remove_container_url(ContainerName, Force, RemoveVolumes),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("DELETE", Url, <<>>, Headers) of
-        {ok, 204, _Headers, _} ->
-            ok;
-        {ok, 304, _Headers, _} ->
-            {error, <<"container already stopped">>};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec get_containers() -> {ok, Containers :: [map()]} | {error, Reason :: binary()}.
-get_containers() ->
-    Url = "/containers/json?all=true",
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("GET", Url, <<>>, Headers) of
-        {ok, 200, _Headers, ContainersBin} ->
-            Containers = json:decode(ContainersBin),
-            {ok, Containers};
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec inspect_container(ContainerId :: binary()) -> {ok, Json :: map()} | {error, Error :: any()}.
-inspect_container(ContainerId) when is_binary(ContainerId) ->
-    Url = lists:flatten(io_lib:format("/containers/~s/json", [binary_to_list(ContainerId)])),
-    Headers = [
-        {<<"Content-Type">>, <<"application/json">>}
-    ],
-    case docker_client:request("GET", Url, <<>>, Headers) of
-        {ok, 200, _Headers, Resp} ->
-            decode_container_inspect_summary(Resp);
-        {ok, _StatusCode, _Header, ErrorResp} ->
-            case catch json:decode(ErrorResp) of
-                #{<<"message">> := Msg} ->
-                    {error, Msg};
-                _ ->
-                    {error, ErrorResp}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec decode_container_inspect_summary(binary()) -> {ok, map()} | {error, binary()}.
-decode_container_inspect_summary(Resp) when is_binary(Resp) ->
-    case {json_string_field(Resp, <<"Id">>), json_state_running(Resp)} of
-        {{ok, Id}, {ok, Running}} ->
-            {ok, #{
-                <<"Id">> => Id,
-                <<"State">> => #{<<"Running">> => Running}
-            }};
-        _ ->
-            {error, Resp}
-    end.
-
--spec json_string_field(binary(), binary()) -> {ok, binary()} | error.
-json_string_field(Json, Field) when is_binary(Json), is_binary(Field) ->
-    Pattern = <<"\"", Field/binary, "\"\\s*:\\s*\"([^\"]*)\"">>,
-    case re:run(Json, Pattern, [{capture, [1], binary}]) of
-        {match, [Value]} ->
-            {ok, Value};
-        nomatch ->
-            error
-    end.
-
--spec json_state_running(binary()) -> {ok, boolean()} | error.
-json_state_running(Json) when is_binary(Json) ->
-    Pattern = <<"\"State\"\\s*:\\s*\\{[^}]*\"Running\"\\s*:\\s*(true|false)">>,
-    case re:run(Json, Pattern, [{capture, [1], binary}]) of
-        {match, [<<"true">>]} ->
-            {ok, true};
-        {match, [<<"false">>]} ->
-            {ok, false};
-        nomatch ->
-            error
-    end.
-
--spec display_options(Options :: map()) -> ok.
-display_options(Options) when is_map(Options) ->
-    logger:debug("deploy options: ~p", [iolist_to_binary(json:encode(Options))]),
-    %lists:foreach(fun({K, V}) -> logger:debug("~p => ~p", [K, V]) end, maps:to_list(Options)),
-    ok.
-
--spec build_stop_container_url(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> string().
-build_stop_container_url(ContainerName, 0) ->
-    lists:flatten(io_lib:format("/containers/~s/stop", [binary_to_list(ContainerName)]));
-build_stop_container_url(ContainerName, TimeoutSeconds) ->
-    lists:flatten(io_lib:format("/containers/~s/stop?t=~B", [binary_to_list(ContainerName), TimeoutSeconds])).
-
--spec build_kill_container_url(ContainerName :: binary(), Signal :: binary()) -> string().
-build_kill_container_url(ContainerName, <<>>) ->
-    lists:flatten(io_lib:format("/containers/~s/kill", [binary_to_list(ContainerName)]));
-build_kill_container_url(ContainerName, Signal) ->
-    lists:flatten(io_lib:format("/containers/~s/kill?signal=~s", [binary_to_list(ContainerName), binary_to_list(Signal)])).
-
--spec build_remove_container_url(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> string().
-build_remove_container_url(ContainerName, Force, RemoveVolumes) ->
-    ForceValue = boolean_to_query_value(Force),
-    RemoveVolumesValue = boolean_to_query_value(RemoveVolumes),
-    lists:flatten(io_lib:format("/containers/~s?force=~s&v=~s", [
-        binary_to_list(ContainerName),
-        ForceValue,
-        RemoveVolumesValue
-    ])).
-
--spec boolean_to_query_value(boolean()) -> string().
-boolean_to_query_value(true) ->
-    "true";
-boolean_to_query_value(false) ->
-    "false".

apps/docker/src/docker_commands_tests.erl

@@ -1,321 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 23. 9月 2025 17:23
-%%%-------------------------------------------------------------------
--module(docker_commands_tests).
--author("anlicheng").
-
-%% API
--export([
-    test_all/0,
-    test_pull/0,
-    test_check_image_exist/0,
-    test_check_image_not_exist/0,
-    test_create_container/0,
-    test_create_container_without_create_options/0,
-    test_create_container_patches_options/0,
-    test_check_container_exist/0,
-    test_check_container_not_exist/0,
-    test_start_container/0,
-    test_stop_container/0,
-    test_stop_container_with_timeout/0,
-    test_kill_container/0,
-    test_kill_container_with_signal/0,
-    test_remove_container/0,
-    test_remove_container_with_options/0,
-    test_get_containers/0
-]).
-
--define(TEST_IMAGE, <<"docker.1ms.run/library/nginx:latest">>).
--define(TEST_CMD, [<<"nginx">>, <<"-g">>, <<"daemon off;">>]).
-
--spec test_all() -> ok.
-test_all() ->
-    ok = test_pull(),
-    ok = test_check_image_exist(),
-    ok = test_check_image_not_exist(),
-    ok = test_create_container(),
-    ok = test_create_container_without_create_options(),
-    ok = test_create_container_patches_options(),
-    ok = test_check_container_exist(),
-    ok = test_check_container_not_exist(),
-    ok = test_start_container(),
-    ok = test_stop_container(),
-    ok = test_stop_container_with_timeout(),
-    ok = test_kill_container(),
-    ok = test_kill_container_with_signal(),
-    ok = test_remove_container(),
-    ok = test_remove_container_with_options(),
-    ok = test_get_containers(),
-    ok.
-
--spec test_pull() -> ok.
-test_pull() ->
-    {ok, Ref, Pid, MRef} = docker_commands:pull_image(?TEST_IMAGE),
-    await_pull(Ref, Pid, MRef).
-
--spec await_pull(reference(), pid(), reference()) -> ok.
-await_pull(Ref, Pid, MRef) ->
-    receive
-        {docker_client, Ref, {response, _Status, _Headers}} ->
-            await_pull(Ref, Pid, MRef);
-        {docker_client, Ref, {data, Data}} ->
-            logger:debug("msg is: ~p", [Data]),
-            await_pull(Ref, Pid, MRef);
-        {docker_client, Ref, done} ->
-            erlang:demonitor(MRef, [flush]),
-            ok;
-        {docker_client, Ref, {error, Reason}} ->
-            erlang:demonitor(MRef, [flush]),
-            error({pull_failed, Reason});
-        {'DOWN', MRef, process, Pid, Reason} ->
-            error({pull_client_down, Reason})
-    end.
-
--spec test_check_image_exist() -> ok.
-test_check_image_exist() ->
-    true = docker_commands:check_image_exist(?TEST_IMAGE),
-    ok.
-
--spec test_check_image_not_exist() -> ok.
-test_check_image_not_exist() ->
-    false = docker_commands:check_image_exist(<<"docker.1ms.run/library/not-exists-for-efka-tests:latest">>),
-    ok.
-
--spec test_create_container() -> ok.
-test_create_container() ->
-    Name = test_container_name(<<"create">>),
-    ContainerDir = prepare_container_dir(Name),
-    {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)),
-    true = is_binary(ContainerId),
-    ok.
-
--spec test_create_container_without_create_options() -> ok.
-test_create_container_without_create_options() ->
-    Name = test_container_name(<<"create-default">>),
-    ContainerDir = prepare_container_dir(Name),
-    Options = docker_container_builder:build_options(Name, ContainerDir, undefined),
-    assert_patched_default_options(Name, ContainerDir, Options),
-    ok.
-
--spec test_create_container_patches_options() -> ok.
-test_create_container_patches_options() ->
-    Name = test_container_name(<<"create-patch">>),
-    ContainerDir = prepare_container_dir(Name),
-    Create = #{
-        config => #{
-            image => ?TEST_IMAGE,
-            cmd => ?TEST_CMD,
-            env => [<<"EXISTING_ENV=1">>],
-            volumes => [<<"/data">>]
-        },
-        host_config => #{
-            binds => [<<"/tmp:/tmp">>]
-        }
-    },
-    Params = #{
-        container_name => Name,
-        create => Create
-    },
-    Options = docker_container_builder:build_options(Name, ContainerDir, Create),
-    assert_patched_default_options(Name, ContainerDir, Options),
-    assert_existing_options_preserved(Options),
-    try
-        ok = test_pull(),
-        {ok, _ContainerId} = docker_commands:create_container(ContainerDir, Params),
-        ok
-    after
-        ok
-        %cleanup_container(Name)
-    end.
-
--spec test_check_container_exist() -> ok.
-test_check_container_exist() ->
-    Name = test_container_name(<<"exist">>),
-    with_created_container(Name, fun(_ContainerDir, _ContainerId) ->
-        true = docker_commands:check_container_exist(Name),
-        ok
-    end).
-
--spec test_check_container_not_exist() -> ok.
-test_check_container_not_exist() ->
-    false = docker_commands:check_container_exist(test_container_name(<<"missing">>)),
-    ok.
-
--spec test_start_container() -> ok.
-test_start_container() ->
-    Name = test_container_name(<<"start">>),
-    with_created_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:start_container(Name),
-        true = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_stop_container() -> ok.
-test_stop_container() ->
-    Name = test_container_name(<<"stop">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:stop_container(Name),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_stop_container_with_timeout() -> ok.
-test_stop_container_with_timeout() ->
-    Name = test_container_name(<<"stop-timeout">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:stop_container(Name, 1),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_kill_container() -> ok.
-test_kill_container() ->
-    Name = test_container_name(<<"kill">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:kill_container(Name),
-        timer:sleep(200),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_kill_container_with_signal() -> ok.
-test_kill_container_with_signal() ->
-    Name = test_container_name(<<"kill-signal">>),
-    with_started_container(Name, fun(_ContainerDir, ContainerId) ->
-        ok = docker_commands:kill_container(Name, <<"SIGKILL">>),
-        timer:sleep(200),
-        false = docker_commands:is_container_running(ContainerId),
-        ok
-    end).
-
--spec test_remove_container() -> ok.
-test_remove_container() ->
-    Name = test_container_name(<<"remove">>),
-    with_created_container(Name, fun(_ContainerDir, _ContainerId) ->
-        ok = docker_commands:remove_container(Name),
-        false = docker_commands:check_container_exist(Name),
-        ok
-    end, false).
-
--spec test_remove_container_with_options() -> ok.
-test_remove_container_with_options() ->
-    Name = test_container_name(<<"remove-opts">>),
-    with_started_container(Name, fun(_ContainerDir, _ContainerId) ->
-        ok = docker_commands:remove_container(Name, true, false),
-        false = docker_commands:check_container_exist(Name),
-        ok
-    end, false).
-
--spec test_get_containers() -> ok.
-test_get_containers() ->
-    Name = test_container_name(<<"list">>),
-    with_created_container(Name, fun(_ContainerDir, ContainerId) ->
-        {ok, Containers} = docker_commands:get_containers(),
-        logger:debug("list containers: ~p", [Containers]),
-        true = is_list(Containers),
-        true = contains_container(Name, ContainerId, Containers),
-        ok
-    end).
-
--spec with_created_container(binary(), fun((string(), binary()) -> ok)) -> ok.
-with_created_container(Name, Fun) ->
-    with_created_container(Name, Fun, true).
-
--spec with_created_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok.
-with_created_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) ->
-    ContainerDir = prepare_container_dir(Name),
-    try
-        ok = test_pull(),
-        {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)),
-        ok = Fun(ContainerDir, ContainerId)
-    after
-        case Cleanup of
-            true ->
-                cleanup_container(Name);
-            false ->
-                ok
-        end
-    end.
-
--spec with_started_container(binary(), fun((string(), binary()) -> ok)) -> ok.
-with_started_container(Name, Fun) ->
-    with_started_container(Name, Fun, true).
-
--spec with_started_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok.
-with_started_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) ->
-    with_created_container(Name, fun(ContainerDir, ContainerId) ->
-        ok = docker_commands:start_container(Name),
-        ok = Fun(ContainerDir, ContainerId)
-    end, Cleanup).
-
--spec minimal_params(binary()) -> map().
-minimal_params(Name) when is_binary(Name) ->
-    #{
-        container_name => Name,
-        create => #{
-            config => #{
-                image => ?TEST_IMAGE,
-                cmd => ?TEST_CMD
-            }
-        }
-    }.
-
--spec prepare_container_dir(binary()) -> string().
-prepare_container_dir(Name) when is_binary(Name) ->
-    Dir = lists:flatten(io_lib:format("/usr/local/code/efka/~ts/", [Name])),
-    ok = filelib:ensure_dir(Dir ++ "placeholder"),
-    ok = file:write_file(Dir ++ "service.conf", <<>>, [write]),
-    Dir.
-
--spec cleanup_container(binary()) -> ok.
-cleanup_container(Name) when is_binary(Name) ->
-    _ = docker_commands:remove_container(Name, true, false),
-    ok.
-
--spec assert_patched_default_options(binary(), string(), map()) -> ok.
-assert_patched_default_options(Name, ContainerDir, Options)
-  when is_binary(Name), is_list(ContainerDir), is_map(Options) ->
-    ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)),
-    ExpectedBind = <<ConfigFile/binary, ":/usr/local/etc/service.conf">>,
-    #{<<"Env">> := Env,
-        <<"Volumes">> := Volumes,
-        <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options,
-    true = lists:member(<<"CONTAINER_NAME=", Name/binary>>, Env),
-    true = maps:is_key(<<"/usr/local/etc/service.conf">>, Volumes),
-    true = lists:member(ExpectedBind, Binds),
-    ok.
-
--spec assert_existing_options_preserved(map()) -> ok.
-assert_existing_options_preserved(Options) when is_map(Options) ->
-    #{<<"Env">> := Env,
-        <<"Volumes">> := Volumes,
-        <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options,
-    true = lists:member(<<"EXISTING_ENV=1">>, Env),
-    true = maps:is_key(<<"/data">>, Volumes),
-    true = lists:member(<<"/tmp:/tmp">>, Binds),
-    ok.
-
--spec contains_container(binary(), binary(), [map()]) -> boolean().
-contains_container(Name, ContainerId, Containers) when is_binary(Name), is_binary(ContainerId), is_list(Containers) ->
-    lists:any(fun(Container) -> container_matches(Name, ContainerId, Container) end, Containers).
-
--spec container_matches(binary(), binary(), map()) -> boolean().
-container_matches(Name, ContainerId, #{<<"Id">> := Id, <<"Names">> := Names}) when is_binary(Id), is_list(Names) ->
-    lists:member(<<"/", Name/binary>>, Names) orelse has_id_prefix(ContainerId, Id);
-container_matches(_Name, _ContainerId, _Container) ->
-    false.
-
--spec has_id_prefix(binary(), binary()) -> boolean().
-has_id_prefix(ExpectedId, ActualId) when is_binary(ExpectedId), is_binary(ActualId) ->
-    PrefixLen = erlang:min(byte_size(ExpectedId), byte_size(ActualId)),
-    binary:part(ExpectedId, 0, PrefixLen) =:= binary:part(ActualId, 0, PrefixLen).
-
--spec test_container_name(binary()) -> binary().
-test_container_name(Prefix) when is_binary(Prefix) ->
-    Suffix = integer_to_binary(erlang:unique_integer([positive])),
-    <<"efka-test-", Prefix/binary, "-", Suffix/binary>>.

apps/docker/src/docker_container_builder.erl

@@ -1,379 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 21. 4月 2026 15:58
-%%%-------------------------------------------------------------------
--module(docker_container_builder).
--author("anlicheng").
-
-%% API
--export([build_options/3]).
-
--spec build_options(binary(), string(), map() | undefined) -> map().
-build_options(ContainerName, ContainerDir, Create0) when is_binary(ContainerName), is_list(ContainerDir) ->
-    ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)),
-    Create = patch_create_options(ContainerName, ConfigFile, Create0),
-    build_create_options(Create).
-
--spec patch_create_options(binary(), binary(), map() | undefined) -> map().
-patch_create_options(ContainerName, ConfigFile, undefined) ->
-    patch_create_options(ContainerName, ConfigFile, #{});
-patch_create_options(ContainerName, ConfigFile, Create0)
-  when is_binary(ContainerName), is_binary(ConfigFile), is_map(Create0) ->
-    Config = patch_container_config(ContainerName, ensure_container_config(field(Create0, <<"config">>, undefined))),
-    HostConfig = patch_host_config(ConfigFile, ensure_host_config(field(Create0, <<"host_config">>, undefined))),
-    Create0#{
-        <<"config">> => Config,
-        <<"host_config">> => HostConfig
-    }.
-
--spec patch_container_config(binary(), map()) -> map().
-patch_container_config(ContainerName, Config0) when is_binary(ContainerName), is_map(Config0) ->
-    Env0 = [to_binary(EnvItem) || EnvItem <- field(Config0, <<"env">>, [])],
-    Volumes0 = [to_binary(Volume) || Volume <- field(Config0, <<"volumes">>, [])],
-    ConfigVolume = <<"/usr/local/etc/service.conf">>,
-    Envs = add_unique_front([<<"CONTAINER_NAME=", ContainerName/binary>>], Env0),
-    Volumes = add_unique_front([ConfigVolume], Volumes0),
-    Config0#{
-        <<"env">> => Envs,
-        <<"volumes">> => Volumes
-    }.
-
--spec patch_host_config(binary(), map()) -> map().
-patch_host_config(ConfigFile, HostConfig0) when is_binary(ConfigFile), is_map(HostConfig0) ->
-    Binds0 = [to_binary(Bind) || Bind <- field(HostConfig0, <<"binds">>, [])],
-    ConfigBind = <<ConfigFile/binary, ":/usr/local/etc/service.conf">>,
-    HostConfig0#{<<"binds">> => add_unique_front([ConfigBind], Binds0)}.
-
--spec ensure_container_config(map() | undefined) -> map().
-ensure_container_config(undefined) ->
-    #{};
-ensure_container_config(Config) when is_map(Config) ->
-    Config.
-
--spec ensure_host_config(map() | undefined) -> map().
-ensure_host_config(undefined) ->
-    #{};
-ensure_host_config(HostConfig) when is_map(HostConfig) ->
-    HostConfig.
-
--spec build_create_options(map()) -> map().
-build_create_options(Create0) when is_map(Create0) ->
-    Config = field(Create0, <<"config">>, #{}),
-    HostConfig = field(Create0, <<"host_config">>, #{}),
-    Endpoints = networking_config_endpoints(field(Create0, <<"networking_config">>, undefined)),
-    #{
-        <<"Image">> => to_binary(field(Config, <<"image">>, <<>>)),
-        <<"Cmd">> => [to_binary(CommandItem) || CommandItem <- field(Config, <<"cmd">>, [])],
-        <<"Entrypoint">> => [to_binary(EntrypointItem) || EntrypointItem <- field(Config, <<"entrypoint">>, [])],
-        <<"Env">> => [to_binary(EnvItem) || EnvItem <- field(Config, <<"env">>, [])],
-        <<"Labels">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(field(Config, <<"labels">>, #{}))]),
-        <<"Volumes">> => build_volumes(field(Config, <<"volumes">>, [])),
-        <<"User">> => to_binary(field(Config, <<"user">>, <<>>)),
-        <<"WorkingDir">> => to_binary(field(Config, <<"working_dir">>, <<>>)),
-        <<"Hostname">> => to_binary(field(Config, <<"hostname">>, <<>>)),
-        <<"ExposedPorts">> => build_expose(field(Config, <<"exposed_ports">>, [])),
-        <<"NetworkingConfig">> => build_networking_config(Endpoints),
-        <<"Healthcheck">> => build_healthcheck(field(Config, <<"healthcheck">>, undefined)),
-        <<"HostConfig">> => fold_merge([
-            build_binds(field(HostConfig, <<"binds">>, [])),
-            build_network_mode(field(HostConfig, <<"network_mode">>, <<>>)),
-            build_restart(field(HostConfig, <<"restart_policy">>, undefined)),
-            build_privileged(field(HostConfig, <<"privileged">>, false)),
-            build_cap_add_drop(field(HostConfig, <<"cap_add">>, []), field(HostConfig, <<"cap_drop">>, [])),
-            build_devices(field(HostConfig, <<"devices">>, [])),
-            build_resources(
-                field(HostConfig, <<"memory">>, 0),
-                field(HostConfig, <<"memory_reservation">>, 0),
-                field(HostConfig, <<"nano_cpus">>, 0),
-                field(HostConfig, <<"cpu_shares">>, 0)
-            ),
-            build_port_bindings(field(HostConfig, <<"port_bindings">>, [])),
-            build_ulimits(field(HostConfig, <<"ulimits">>, [])),
-            build_tmpfs(field(HostConfig, <<"tmpfs">>, #{})),
-            build_sysctls(field(HostConfig, <<"sysctls">>, #{})),
-            build_extra_hosts(field(HostConfig, <<"extra_hosts">>, []))
-        ])
-    }.
-
--spec networking_config_endpoints(map() | undefined) -> [map()].
-networking_config_endpoints(undefined) ->
-    [];
-networking_config_endpoints(NetworkingConfig) when is_map(NetworkingConfig) ->
-    field(NetworkingConfig, <<"endpoints">>, []).
-
--spec fold_merge([map()]) -> map().
-fold_merge(List) ->
-    lists:foldl(fun maps:merge/2, #{}, List).
-
--spec build_expose([map()]) -> map().
-build_expose(Ports) when is_list(Ports) ->
-    case Ports of
-        [] ->
-            #{};
-        _ ->
-            maps:from_list([{normalize_expose_port(Port), #{}} || Port <- Ports])
-    end.
-
--spec build_volumes([binary() | list() | atom()]) -> map().
-build_volumes(Volumes) when is_list(Volumes) ->
-    case Volumes of
-        [] ->
-            #{};
-        _ ->
-            maps:from_list([{to_binary(ContainerPath), #{}} || ContainerPath <- Volumes])
-    end.
-
--spec build_binds([binary() | list() | atom()]) -> map().
-build_binds(Binds) when is_list(Binds) ->
-    case Binds of
-        [] ->
-            #{};
-        _ ->
-            #{<<"Binds">> => [to_binary(Bind) || Bind <- Binds]}
-    end.
-
--spec build_networking_config([map()]) -> map().
-build_networking_config(Endpoints) when is_list(Endpoints) ->
-    case Endpoints of
-        [] ->
-            #{};
-        _ ->
-            NetCfg = maps:from_list([{to_binary(Name), #{}} || #{<<"name">> := Name} <- Endpoints]),
-            #{<<"EndpointsConfig">> => NetCfg}
-    end.
-
--spec build_network_mode(binary() | list() | atom()) -> map().
-build_network_mode(<<>>) ->
-    #{};
-build_network_mode(NetworkMode) ->
-    #{<<"NetworkMode">> => to_binary(NetworkMode)}.
-
--spec build_healthcheck(map() | undefined) -> map().
-build_healthcheck(undefined) ->
-    #{};
-build_healthcheck(Healthcheck) when is_map(Healthcheck) ->
-    #{
-        <<"Test">> => [to_binary(Item) || Item <- field(Healthcheck, <<"test">>, [])],
-        <<"Interval">> => field(Healthcheck, <<"interval_ns">>, 0),
-        <<"Timeout">> => field(Healthcheck, <<"timeout_ns">>, 0),
-        <<"Retries">> => field(Healthcheck, <<"retries">>, 0)
-    }.
-
--spec build_restart(map() | undefined) -> map().
-build_restart(undefined) ->
-    #{};
-build_restart(RestartPolicy0) when is_map(RestartPolicy0) ->
-    RestartPolicy = #{
-        <<"Name">> => to_binary(field(RestartPolicy0, <<"name">>, <<>>))
-    },
-    case field(RestartPolicy0, <<"maximum_retry_count">>, 0) of
-        0 ->
-            #{<<"RestartPolicy">> => RestartPolicy};
-        RetryCount ->
-            #{<<"RestartPolicy">> => RestartPolicy#{
-                <<"MaximumRetryCount">> => RetryCount
-            }}
-    end.
-
--spec build_privileged(any()) -> map().
-build_privileged(Privileged) ->
-    case to_bool(Privileged) of
-        true ->
-            #{<<"Privileged">> => true};
-        _ ->
-            #{}
-    end.
-
--spec build_cap_add_drop([binary() | list() | atom()], [binary() | list() | atom()]) -> map().
-build_cap_add_drop(Add, Drop) when is_list(Add), is_list(Drop) ->
-    case {Add, Drop} of
-        {[], []} ->
-            #{};
-        _ ->
-            #{
-                <<"CapAdd">> => [to_binary(Item) || Item <- Add],
-                <<"CapDrop">> => [to_binary(Item) || Item <- Drop]
-            }
-    end.
-
--spec build_devices([map()]) -> map().
-build_devices(Devices) when is_list(Devices) ->
-    case Devices of
-        [] ->
-            #{};
-        _ ->
-            DevObjs = [#{
-                <<"PathOnHost">> => to_binary(HostPath),
-                <<"PathInContainer">> => to_binary(ContainerPath),
-                <<"CgroupPermissions">> => device_permissions(Permissions)
-            } || #{
-                <<"path_on_host">> := HostPath,
-                <<"path_in_container">> := ContainerPath,
-                <<"cgroup_permissions">> := Permissions
-            } <- Devices],
-            #{<<"Devices">> => DevObjs}
-    end.
-
--spec build_resources(integer(), integer(), integer(), integer()) -> map().
-build_resources(MemoryBytes, ReservationBytes, NanoCpus, CpuShares) ->
-    HostConfig0 = #{},
-    HostConfig1 = case MemoryBytes of
-        0 ->
-            HostConfig0;
-        _ ->
-            HostConfig0#{<<"Memory">> => MemoryBytes}
-    end,
-    HostConfig2 = case ReservationBytes of
-        0 ->
-            HostConfig1;
-        _ ->
-            HostConfig1#{<<"MemoryReservation">> => ReservationBytes}
-    end,
-    HostConfig3 = case NanoCpus of
-        0 ->
-            HostConfig2;
-        _ ->
-            HostConfig2#{<<"NanoCpus">> => NanoCpus}
-    end,
-    case CpuShares of
-        0 ->
-            HostConfig3;
-        _ ->
-            HostConfig3#{<<"CpuShares">> => CpuShares}
-    end.
-
--spec build_port_bindings([map()]) -> map().
-build_port_bindings(PortBindings) when is_list(PortBindings) ->
-    case PortBindings of
-        [] ->
-            #{};
-        _ ->
-            #{<<"PortBindings">> => fold_port_bindings(PortBindings)}
-    end.
-
--spec fold_port_bindings([map()]) -> map().
-fold_port_bindings(PortBindings) ->
-    lists:foldl(
-        fun(PortBinding, Acc) ->
-            PortKey = port_binding_key(PortBinding),
-            Binding = port_binding_value(PortBinding),
-            Existing = maps:get(PortKey, Acc, []),
-            Acc#{PortKey => Existing ++ [Binding]}
-        end,
-        #{},
-        PortBindings).
-
--spec port_binding_key(map()) -> binary().
-port_binding_key(#{<<"container_port">> := ContainerPort, <<"protocol">> := Protocol}) ->
-    PortBin = integer_to_binary(ContainerPort),
-    ProtocolBin = to_binary(Protocol),
-    case ProtocolBin of
-        <<>> ->
-            <<PortBin/binary, "/tcp">>;
-        <<"tcp">> ->
-            <<PortBin/binary, "/tcp">>;
-        _ ->
-            <<PortBin/binary, "/", ProtocolBin/binary>>
-    end.
-
--spec port_binding_value(map()) -> map().
-port_binding_value(#{<<"host_port">> := HostPort} = PortBinding) ->
-    HostIp = to_binary(field(PortBinding, <<"host_ip">>, <<>>)),
-    #{<<"HostIp">> => HostIp, <<"HostPort">> => integer_to_binary(HostPort)}.
-
--spec build_ulimits([map()]) -> map().
-build_ulimits(Ulimits) when is_list(Ulimits) ->
-    case Ulimits of
-        [] ->
-            #{};
-        _ ->
-            #{<<"Ulimits">> => [#{
-                <<"Name">> => to_binary(Name),
-                <<"Soft">> => Soft,
-                <<"Hard">> => Hard
-            } || #{<<"name">> := Name, <<"soft">> := Soft, <<"hard">> := Hard} <- Ulimits]}
-    end.
-
--spec build_sysctls(map()) -> map().
-build_sysctls(Sysctls) when is_map(Sysctls) ->
-    case maps:size(Sysctls) of
-        0 ->
-            #{};
-        _ ->
-            #{<<"Sysctls">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(Sysctls)])}
-    end.
-
--spec build_tmpfs(map()) -> map().
-build_tmpfs(Tmpfs) when is_map(Tmpfs) ->
-    case maps:size(Tmpfs) of
-        0 ->
-            #{};
-        _ ->
-            #{<<"Tmpfs">> => maps:from_list([{to_binary(Path), to_binary(Options)} || {Path, Options} <- maps:to_list(Tmpfs)])}
-    end.
-
--spec build_extra_hosts([binary() | list() | atom()]) -> map().
-build_extra_hosts(Hosts) when is_list(Hosts) ->
-    case Hosts of
-        [] ->
-            #{};
-        _ ->
-            #{<<"ExtraHosts">> => [to_binary(Host) || Host <- Hosts]}
-    end.
-
--spec normalize_expose_port(map()) -> binary().
-normalize_expose_port(#{<<"container_port">> := Port, <<"protocol">> := Protocol}) ->
-    PortBin = integer_to_binary(Port),
-    ProtocolBin = to_binary(Protocol),
-    case ProtocolBin of
-        <<>> ->
-            <<PortBin/binary, "/tcp">>;
-        <<"tcp">> ->
-            <<PortBin/binary, "/tcp">>;
-        _ ->
-            <<PortBin/binary, "/", ProtocolBin/binary>>
-    end.
-
--spec device_permissions(binary() | list() | atom()) -> binary().
-device_permissions(<<>>) ->
-    <<"rwm">>;
-device_permissions(Permissions) ->
-    to_binary(Permissions).
-
--spec add_unique_front([binary()], [binary()]) -> [binary()].
-add_unique_front([], List) ->
-    List;
-add_unique_front([Item | Rest], List) ->
-    NList = case lists:member(Item, List) of
-        true ->
-            List;
-        false ->
-            [Item | List]
-    end,
-    add_unique_front(Rest, NList).
-
--spec field(map(), binary(), term()) -> term().
-field(Map, Key, Default) when is_map(Map), is_binary(Key) ->
-    maps:get(Key, Map, Default).
-
--spec to_binary(binary() | list() | atom() | any()) -> binary().
-to_binary(Value) when is_binary(Value) ->
-    Value;
-to_binary(Value) when is_list(Value) ->
-    list_to_binary(Value);
-to_binary(Value) when is_atom(Value) ->
-    atom_to_binary(Value, utf8);
-to_binary(Value) ->
-    iolist_to_binary(io_lib:format("~p", [Value])).
-
--spec to_bool(true | false | 0 | 1 | undefined) -> boolean().
-to_bool(true) ->
-    true;
-to_bool(1) ->
-    true;
-to_bool(_) ->
-    false.

apps/docker/src/docker_deployer.erl

@@ -1,233 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 07. 5月 2025 15:47
-%%%-------------------------------------------------------------------
--module(docker_deployer).
--author("anlicheng").
--dialyzer([{nowarn_function, normalize_image/1}]).
-
-%% API
--export([deploy/4]).
-
--define(TASK_SUCCESS, <<"success">>).
--define(TASK_FAIL, <<"fail">>).
-
--type reporter() :: {stream, pos_integer()}.
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
-%{
-%   "image": "nginx:latest",
-%   "container_name": "my_nginx",
-%   "ports": ["8080:80", "443:443"],
-%   "volumes": ["/host/data:/data", "/host/log:/var/log"],
-%   "envs": ["ENV1=val1", "ENV2=val2"],
-%   "entrypoint": ["/docker-entrypoint.sh"],
-%   "command": ["nginx", "-g", "daemon off;"],
-%   "restart": "always"
-%}
--spec deploy(TaskId :: integer(), ContainerDir :: string(), Params :: map(), Reporter :: reporter()) -> ok.
-deploy(TaskId, ContainerDir, Params, Reporter) when is_integer(TaskId), is_list(ContainerDir), is_map(Params) ->
-    ContainerName = deploy_container_name(Params),
-    Image0 = deploy_image(Params),
-    report_stream_event(Reporter, TaskId, <<"info">>, <<"开始部署容器:"/utf8, ContainerName/binary>>),
-    try
-        ok = ensure_container_absent(Reporter, TaskId, ContainerName),
-        {ok, Image} = ensure_image_ready(Reporter, TaskId, Image0),
-        {ok, ContainerId} = create_container_and_config(Reporter, TaskId, ContainerDir, Params),
-        ShortContainerId = short_container_id(ContainerId),
-        report_stream_event(Reporter, TaskId, <<"container_id">>, ContainerId),
-        report_stream_event(Reporter, TaskId, <<"info">>, <<"容器创建成功: "/utf8, ShortContainerId/binary>>),
-        report_stream_event(Reporter, TaskId, <<"info">>, <<"任务完成"/utf8>>),
-        write_task_summary(TaskId, <<"success">>, ContainerName, Image, ContainerId),
-        close_task(Reporter, TaskId, ?TASK_SUCCESS)
-    catch
-        throw:{deploy_error, Reason} ->
-            report_stream_event(Reporter, TaskId, <<"error">>, Reason),
-            report_stream_event(Reporter, TaskId, <<"error">>, <<"任务失败"/utf8>>),
-            write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined),
-            write_task_failure_reason(TaskId, Reason),
-            close_task(Reporter, TaskId, ?TASK_FAIL);
-        Class:Reason:Stacktrace ->
-            Error = iolist_to_binary(io_lib:format("deploy crashed: ~p:~p ~p", [Class, Reason, Stacktrace])),
-            report_stream_event(Reporter, TaskId, <<"error">>, Error),
-            report_stream_event(Reporter, TaskId, <<"error">>, <<"任务失败"/utf8>>),
-            write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined),
-            write_task_failure_reason(TaskId, Error),
-            close_task(Reporter, TaskId, ?TASK_FAIL)
-    end.
-
--spec normalize_image(binary()) -> binary().
-normalize_image(Image) when is_binary(Image) ->
-    Parts = binary:split(Image, <<"/">>, [global]),
-    {PrefixParts, [Last]} = lists:split(length(Parts) - 1, Parts),
-    NormalizedLast = case binary:split(Last, <<":">>, [global]) of
-                         [_Name] -> <<Last/binary, ":latest">>;
-                         [_Name, _Tag] -> Last
-                     end,
-    iolist_to_binary(lists:join(<<"/">>, PrefixParts ++ [NormalizedLast])).
-
--spec report_stream_event(reporter(), TaskId :: integer(), Level :: binary(), Msg :: binary()) -> ok.
-report_stream_event({stream, StreamId}, _TaskId, Level, Msg) when is_integer(StreamId), is_binary(Level), is_binary(Msg) ->
-    efka_iot_client:send_stream(StreamId, {data, encode_stream_event(Level, Msg)}).
-
--spec close_task(reporter(), integer(), binary()) -> ok.
-close_task({stream, StreamId}, _TaskId, Reason) when is_integer(StreamId), is_binary(Reason) ->
-    ok = efka_iot_client:send_stream(StreamId, {data, encode_close_event(Reason)}),
-    efka_iot_client:send_stream(StreamId, fin).
-
--spec encode_stream_event(binary(), binary()) -> binary().
-encode_stream_event(<<"container_id">>, ContainerId) ->
-    iolist_to_binary(json:encode(#{
-        <<"type">> => <<"container_id">>,
-        <<"container_id">> => ContainerId
-    }));
-encode_stream_event(Type, Msg) ->
-    iolist_to_binary(json:encode(#{
-        <<"type">> => Type,
-        <<"message">> => Msg
-    })).
-
--spec encode_close_event(binary()) -> binary().
-encode_close_event(Reason) ->
-    iolist_to_binary(json:encode(#{
-        <<"type">> => <<"close">>,
-        <<"reason">> => Reason
-    })).
-
--spec write_task_summary(integer(), binary(), binary(), binary(), undefined | binary()) -> ok.
-write_task_summary(TaskId, Status, ContainerName, Image, ContainerId)
-  when is_integer(TaskId), is_binary(Status), is_binary(ContainerName), is_binary(Image) ->
-    Fields0 = [
-        <<"type=deploy_summary">>,
-        <<"task_id=">>, integer_to_binary(TaskId),
-        <<" status=">>, Status,
-        <<" container_name=">>, ContainerName,
-        <<" image=">>, Image
-    ],
-    Fields = case ContainerId of
-                 undefined ->
-                     Fields0;
-                 ContainerId0 when is_binary(ContainerId0) ->
-                     Fields0 ++ [<<" container_id=">>, short_container_id(ContainerId0)]
-             end,
-    efka_logger:write(iolist_to_binary(Fields)).
-
--spec write_task_failure_reason(integer(), binary()) -> ok.
-write_task_failure_reason(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) ->
-    Info = iolist_to_binary([
-        <<"type=deploy_failure_reason task_id=">>,
-        integer_to_binary(TaskId),
-        <<" reason=">>,
-        Reason
-    ]),
-    efka_logger:write(Info).
-
--spec ensure_container_absent(reporter(), TaskId :: integer(), ContainerName :: binary()) -> ok.
-ensure_container_absent(Reporter, TaskId, ContainerName) when is_integer(TaskId), is_binary(ContainerName) ->
-    report_stream_event(Reporter, TaskId, <<"info">>, <<"开始创建容器: "/utf8, ContainerName/binary>>),
-    ok.
-
--spec ensure_image_ready(reporter(), TaskId :: integer(), Image0 :: binary()) -> {ok, binary()}.
-ensure_image_ready(Reporter, TaskId, Image0) when is_integer(TaskId), is_binary(Image0) ->
-    Image = normalize_image(Image0),
-    report_stream_event(Reporter, TaskId, <<"info">>, <<"使用镜像:"/utf8, Image/binary>>),
-    case docker_commands:check_image_exist(Image) of
-        true ->
-            report_stream_event(Reporter, TaskId, <<"info">>, <<"本地镜像已存在，跳过拉取:"/utf8, Image/binary>>),
-            {ok, Image};
-        false ->
-            report_stream_event(Reporter, TaskId, <<"info">>, <<"开始拉取镜像:"/utf8, Image/binary>>),
-            case docker_commands:pull_image(Image) of
-                {ok, Ref, Pid, MRef} ->
-                    await_pull_image(Reporter, TaskId, Ref, Pid, MRef),
-                    {ok, Image}
-            end
-    end.
-
--spec await_pull_image(reporter(), TaskId :: integer(), Ref :: reference(), Pid :: pid(), MRef :: reference()) -> ok.
-await_pull_image(Reporter, TaskId, Ref, Pid, MRef) ->
-    receive
-        {docker_client, Ref, {response, _Status, _Headers}} ->
-            await_pull_image(Reporter, TaskId, Ref, Pid, MRef);
-        {docker_client, Ref, {data, Data}} ->
-            report_stream_event(Reporter, TaskId, <<"info">>, Data),
-            await_pull_image(Reporter, TaskId, Ref, Pid, MRef);
-        {docker_client, Ref, done} ->
-            erlang:demonitor(MRef, [flush]),
-            ok;
-        {docker_client, Ref, {error, Reason}} ->
-            erlang:demonitor(MRef, [flush]),
-            report_stream_event(Reporter, TaskId, <<"error">>, Reason),
-            throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>});
-        {'DOWN', MRef, process, Pid, Reason0} ->
-            Reason = iolist_to_binary(io_lib:format("~p", [Reason0])),
-            throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>})
-    end.
-
--spec create_container_and_config(reporter(), TaskId :: integer(), ContainerDir :: string(), Params :: map()) ->
-    {ok, binary()}.
-create_container_and_config(Reporter, TaskId, ContainerDir, Params)
-  when is_integer(TaskId), is_list(ContainerDir), is_map(Params) ->
-    case docker_commands:create_container(ContainerDir, Params) of
-        {ok, ContainerId} ->
-            ok = create_config_file(Reporter, TaskId, ContainerDir),
-            {ok, ContainerId};
-        {error, Reason} when is_binary(Reason) ->
-            throw({deploy_error, format_create_container_error(Reason)});
-        {error, Reason} ->
-            Error = iolist_to_binary(io_lib:format("容器创建失败: ~p", [Reason])),
-            throw({deploy_error, Error})
-    end.
-
--spec create_config_file(reporter(), TaskId :: integer(), ContainerDir :: string()) -> ok.
-create_config_file(Reporter, TaskId, ContainerDir) when is_integer(TaskId), is_list(ContainerDir) ->
-    ConfigFile = docker_helper:get_config_file(ContainerDir),
-    case file:open(ConfigFile, [write, exclusive]) of
-        {ok, FD} ->
-            ok = file:write(FD, <<>>),
-            file:close(FD),
-            ok;
-        {error, Reason} ->
-            ReasonBin = list_to_binary(io_lib:format("~p", [Reason])),
-            report_stream_event(Reporter, TaskId, <<"notice">>, <<"创建配置文件失败: "/utf8, ReasonBin/binary>>),
-            ok
-    end.
-
--spec format_create_container_error(binary()) -> binary().
-format_create_container_error(Reason) when is_binary(Reason) ->
-    case is_container_already_exists_error(Reason) of
-        true ->
-            <<"本地容器已经存在"/utf8>>;
-        false ->
-            <<"容器创建失败: "/utf8, Reason/binary>>
-    end.
-
--spec is_container_already_exists_error(binary()) -> boolean().
-is_container_already_exists_error(Reason) when is_binary(Reason) ->
-    binary:match(Reason, <<"is already in use by container">>) =/= nomatch orelse
-        binary:match(Reason, <<"Conflict. The container name ">>) =/= nomatch.
-
--spec short_container_id(binary()) -> binary().
-short_container_id(ContainerId) when is_binary(ContainerId), byte_size(ContainerId) >= 12 ->
-    binary:part(ContainerId, 0, 12);
-short_container_id(ContainerId) when is_binary(ContainerId) ->
-    ContainerId.
-
--spec deploy_container_name(map()) -> binary().
-deploy_container_name(#{<<"container_name">> := ContainerName}) when is_binary(ContainerName) ->
-    ContainerName;
-deploy_container_name(_) ->
-    throw({deploy_error, <<"invalid deploy params: container_name missing">>}).
-
--spec deploy_image(map()) -> binary().
-deploy_image(#{<<"create">> := #{<<"config">> := #{<<"image">> := Image}}}) when is_binary(Image) ->
-    Image;
-deploy_image(_) ->
-    throw({deploy_error, <<"invalid deploy params: image missing">>}).

apps/docker/src/docker_helper.erl

@@ -1,102 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 17. 9月 2025 14:50
-%%%-------------------------------------------------------------------
--module(docker_helper).
--author("anlicheng").
-
-%% API
--export([root_dir/0]).
--export([ensure_container_dir/2, ensure_container_dir/3, get_container_dir/2, get_config_file/1]).
--export([update_container_config/2]).
-
--spec root_dir() -> {ok, string()} | undefined.
-root_dir() ->
-    case application:get_env(docker, root_dir) of
-        {ok, RootDir} ->
-            {ok, RootDir};
-        undefined ->
-            application:get_env(efka, root_dir)
-    end.
-
--spec ensure_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()}.
-ensure_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
-    ensure_container_dir(RootDir, ContainerName, <<>>).
-
--spec ensure_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) ->
-    {ok, ServerRootDir :: string()}.
-ensure_container_dir(RootDir, ContainerName, ContainerDir) when is_list(RootDir), is_binary(ContainerName), is_binary(ContainerDir) ->
-    PointerDir = default_container_dir(RootDir, ContainerName),
-    PointerFile = container_dir_pointer_file(PointerDir),
-    ActualContainerDir = resolve_container_dir(RootDir, ContainerName, ContainerDir),
-    ok = filelib:ensure_dir(PointerDir),
-    ok = filelib:ensure_dir(ActualContainerDir),
-    ok = file:write_file(PointerFile, unicode:characters_to_binary(ActualContainerDir), [write]),
-    {ok, ActualContainerDir}.
-
--spec get_config_file(ContainerDir :: string()) -> ConfigFile :: string().
-get_config_file(ContainerDir) when is_list(ContainerDir) ->
-    %% 根目录
-    ContainerDir ++ "service.conf".
-
--spec get_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()} | error.
-get_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
-    ContainerRootDir = default_container_dir(RootDir, ContainerName),
-    PointerFile = container_dir_pointer_file(ContainerRootDir),
-    ResolvedContainerDir = case file:read_file(PointerFile) of
-        {ok, ActualContainerDirBin} ->
-            normalize_container_dir(binary_to_list(ActualContainerDirBin));
-        {error, _} ->
-            ContainerRootDir
-    end,
-    case filelib:is_dir(ResolvedContainerDir) of
-        true ->
-            {ok, ResolvedContainerDir};
-        false ->
-            error
-    end.
-
--spec update_container_config(binary(), binary()) -> ok | {error, binary()}.
-update_container_config(ContainerName, Config) when is_binary(ContainerName), is_binary(Config) ->
-    {ok, RootDir} = root_dir(),
-    case get_container_dir(RootDir, ContainerName) of
-        {ok, ContainerDir} ->
-            ConfigFile = get_config_file(ContainerDir),
-            case file:write_file(ConfigFile, Config, [write, binary]) of
-                ok ->
-                    logger:warning("[efka_iot_client] write config file: ~p success", [ConfigFile]),
-                    ok;
-                {error, Reason} ->
-                    logger:warning("[efka_iot_client] write config file: ~p, get error: ~p", [ConfigFile, Reason]),
-                    {error, <<"write config failed">>}
-            end;
-        error ->
-            {error, <<"error">>}
-    end.
-
--spec default_container_dir(RootDir :: string(), ContainerName :: binary()) -> string().
-default_container_dir(RootDir, ContainerName) ->
-    normalize_container_dir(RootDir ++ "/" ++ binary_to_list(ContainerName)).
-
--spec container_dir_pointer_file(ContainerDir :: string()) -> string().
-container_dir_pointer_file(ContainerDir) ->
-    ContainerDir ++ ".container_dir".
-
--spec resolve_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) -> string().
-resolve_container_dir(RootDir, ContainerName, <<>>) ->
-    default_container_dir(RootDir, ContainerName);
-resolve_container_dir(_RootDir, _ContainerName, ContainerDir) ->
-    normalize_container_dir(binary_to_list(ContainerDir)).
-
--spec normalize_container_dir(ContainerDir :: string()) -> string().
-normalize_container_dir(ContainerDir) ->
-    case lists:last(ContainerDir) of
-        $/ ->
-            ContainerDir;
-        _ ->
-            ContainerDir ++ "/"
-    end.

apps/docker/src/docker_sup.erl

@@ -1,24 +0,0 @@
-%%%-------------------------------------------------------------------
-%% @doc Docker top level supervisor.
-%% @end
-%%%-------------------------------------------------------------------
-
--module(docker_sup).
-
--behaviour(supervisor).
-
--export([start_link/0]).
--export([init/1]).
-
--define(SERVER, ?MODULE).
-
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
-start_link() ->
-    supervisor:start_link({local, ?SERVER}, ?MODULE, []).
-
--spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
-init([]) ->
-    SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
-    ChildSpecs = [],
-
-    {ok, {SupFlags, ChildSpecs}}.

apps/docker/test/efka_docker_container_builder_tests.erl

@@ -1,35 +0,0 @@
--module(efka_docker_container_builder_tests).
-
--include_lib("eunit/include/eunit.hrl").
-
-port_bindings_are_rendered_for_docker_create_test() ->
-    Name = <<"nginx">>,
-    ContainerDir = "/tmp/efka-nginx/",
-    Create = #{
-        <<"config">> => #{
-            <<"image">> => <<"nginx:latest">>,
-            <<"exposed_ports">> => [
-                #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-                #{<<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
-            ]
-        },
-        <<"host_config">> => #{
-            <<"port_bindings">> => [
-                #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-                #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
-            ]
-        }
-    },
-    Options = docker_container_builder:build_options(Name, ContainerDir, Create),
-    ?assertMatch(#{
-        <<"ExposedPorts">> := #{
-            <<"80/tcp">> := #{},
-            <<"443/tcp">> := #{}
-        },
-        <<"HostConfig">> := #{
-            <<"PortBindings">> := #{
-                <<"80/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"8080">>}],
-                <<"443/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"443">>}]
-            }
-        }
-    }, Options).

apps/efka/include/message.hrl

@@ -1,25 +1,86 @@
 %%%-------------------------------------------------------------------
 %%% @author anlicheng
-%%% @copyright (C) 2026, <COMPANY>
+%%% @copyright (C) 2025, <COMPANY>
 %%% @doc
-%%%
+%%% 扩展部分, 1: 支持基于topic的pub/sub机制; 2. 基于target的单点通讯和广播
 %%% @end
-%%% Created : 07. 7月 2026 17:09
+%%% Created : 21. 4月 2025 17:28
 %%%-------------------------------------------------------------------
 -author("anlicheng").
 
-%%--------------------------------------------------------------------
-%% Wire classes
-%%--------------------------------------------------------------------
--define(CLASS_REQUEST, 1).
--define(CLASS_RESPONSE, 2).
--define(CLASS_COMMAND, 3).
--define(CLASS_COMMAND_RESPONSE, 4).
--define(CLASS_MESSAGE, 5).
--define(CLASS_STREAM, 6).
+%% efka主动发起的消息体类型, 消息大类
+-define(PACKET_REQUEST, 16#01).
+-define(PACKET_RESPONSE, 16#02).
 
-%%--------------------------------------------------------------------
-%% Stream targets
-%%--------------------------------------------------------------------
--define(STREAM_TARGET_MANAGER, 1).
--define(STREAM_TARGET_CONTAINER_DEPLOY, 2).
+%% efka主动发起不需要返回的数据
+-define(PACKET_CAST, 16#03).
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%% 二级分类定义
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+%% 主机端上报数据类型标识
+-define(MESSAGE_AUTH_REQUEST, 16#01).
+-define(MESSAGE_AUTH_REPLY, 16#02).
+
+-define(MESSAGE_COMMAND, 16#03).
+-define(MESSAGE_DEPLOY, 16#04).
+-define(MESSAGE_PUB, 16#05).
+
+-define(MESSAGE_DATA, 16#06).
+-define(MESSAGE_EVENT, 16#07).
+
+%% efka主动上报的event-stream流, 单向消息，主要是: docker-create的实时处理逻辑上报
+-define(MESSAGE_EVENT_STREAM, 16#08).
+
+-define(MESSAGE_JSONRPC_REQUEST, 16#F0).
+-define(MESSAGE_JSONRPC_REPLY, 16#F1).
+
+%%%% 命令类型子分类, 不需要返回值
+%% 授权
+-define(COMMAND_AUTH, 16#08).
+
+-record(auth_request, {
+    uuid :: binary(),
+    username :: binary(),
+    salt :: binary(),
+    token :: binary(),
+    timestamp :: integer()
+}).
+
+-record(auth_reply, {
+    code :: integer(),
+    payload :: binary()
+}).
+
+-record(pub, {
+    topic :: binary(),
+    qos = 0 :: integer(),
+    content :: binary()
+}).
+
+-record(command, {
+    command_type :: integer(),
+    command :: binary()
+}).
+
+-record(jsonrpc_request, {
+    method :: binary(),
+    params = <<>> :: any()
+}).
+
+-record(jsonrpc_reply, {
+    result :: any() | undefined,
+    error :: any() | undefined
+}).
+
+-record(data, {
+    route_key :: binary(),
+    metric :: binary()
+}).
+
+-record(task_event_stream, {
+    task_id :: integer(),
+    type :: binary(),
+    stream :: binary()
+}).

apps/efka/include/message_pb.hrl

@@ -1,225 +0,0 @@
-%% -*- coding: utf-8 -*-
-%% Automatically generated, do not edit
-%% Generated by gpb_compile version 4.21.7
-
--ifndef(message_pb).
--define(message_pb, true).
-
--define(message_pb_gpb_version, "4.21.7").
-
-
--ifndef('REQUEST.AUTHREQUEST_PB_H').
--define('REQUEST.AUTHREQUEST_PB_H', true).
--record('Request.AuthRequest',
-        {uuid = <<>>            :: iodata() | undefined, % = 1, optional
-         token = <<>>           :: iodata() | undefined, % = 2, optional
-         timestamp = 0          :: non_neg_integer() | undefined % = 3, optional, 64 bits
-        }).
--endif.
-
--ifndef('REQUEST_PB_H').
--define('REQUEST_PB_H', true).
--record('Request',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         body                   :: {auth_request, message_pb:'Request.AuthRequest'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('RESPONSE.ERROR_PB_H').
--define('RESPONSE.ERROR_PB_H', true).
--record('Response.Error',
-        {code = 0               :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         reason = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('RESPONSE.AUTHRESPONSE_PB_H').
--define('RESPONSE.AUTHRESPONSE_PB_H', true).
--record('Response.AuthResponse',
-        {
-        }).
--endif.
-
--ifndef('RESPONSE_PB_H').
--define('RESPONSE_PB_H', true).
--record('Response',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         body                   :: {auth_response, message_pb:'Response.AuthResponse'()} | {error, message_pb:'Response.Error'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER_PB_H').
--define('COMMAND.CONTAINER_PB_H', true).
--record('Command.Container',
-        {action                 :: {list, message_pb:'Command.Container.ContainerList'()} | {start, message_pb:'Command.Container.ContainerStart'()} | {stop, message_pb:'Command.Container.ContainerStop'()} | {kill, message_pb:'Command.Container.ContainerKill'()} | {remove, message_pb:'Command.Container.ContainerRemove'()} | {config, message_pb:'Command.Container.ContainerConfig'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERCONFIG_PB_H').
--define('COMMAND.CONTAINER.CONTAINERCONFIG_PB_H', true).
--record('Command.Container.ContainerConfig',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         config = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERREMOVE_PB_H').
--define('COMMAND.CONTAINER.CONTAINERREMOVE_PB_H', true).
--record('Command.Container.ContainerRemove',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         force = false          :: boolean() | 0 | 1 | undefined, % = 2, optional
-         remove_volumes = false :: boolean() | 0 | 1 | undefined % = 3, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERKILL_PB_H').
--define('COMMAND.CONTAINER.CONTAINERKILL_PB_H', true).
--record('Command.Container.ContainerKill',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         signal = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERSTOP_PB_H').
--define('COMMAND.CONTAINER.CONTAINERSTOP_PB_H', true).
--record('Command.Container.ContainerStop',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined, % = 1, optional
-         timeout_seconds = 0    :: non_neg_integer() | undefined % = 2, optional, 32 bits
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERSTART_PB_H').
--define('COMMAND.CONTAINER.CONTAINERSTART_PB_H', true).
--record('Command.Container.ContainerStart',
-        {target = undefined     :: message_pb:'Command.Container.ContainerTarget'() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERLIST_PB_H').
--define('COMMAND.CONTAINER.CONTAINERLIST_PB_H', true).
--record('Command.Container.ContainerList',
-        {
-        }).
--endif.
-
--ifndef('COMMAND.CONTAINER.CONTAINERTARGET_PB_H').
--define('COMMAND.CONTAINER.CONTAINERTARGET_PB_H', true).
--record('Command.Container.ContainerTarget',
-        {name = <<>>            :: iodata() | undefined, % = 1, optional
-         id = <<>>              :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('COMMAND_PB_H').
--define('COMMAND_PB_H', true).
--record('Command',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         body                   :: {container, message_pb:'Command.Container'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('COMMANDRESPONSE.ERROR_PB_H').
--define('COMMANDRESPONSE.ERROR_PB_H', true).
--record('CommandResponse.Error',
-        {code = 0               :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         reason = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('COMMANDRESPONSE_PB_H').
--define('COMMANDRESPONSE_PB_H', true).
--record('CommandResponse',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         body                   :: {result, iodata()} | {error, message_pb:'CommandResponse.Error'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('MESSAGE.PING_PB_H').
--define('MESSAGE.PING_PB_H', true).
--record('Message.Ping',
-        {
-        }).
--endif.
-
--ifndef('MESSAGE.PONG_PB_H').
--define('MESSAGE.PONG_PB_H', true).
--record('Message.Pong',
-        {
-        }).
--endif.
-
--ifndef('MESSAGE.PUB_PB_H').
--define('MESSAGE.PUB_PB_H', true).
--record('Message.Pub',
-        {topic = <<>>           :: iodata() | undefined, % = 1, optional
-         qos = 0                :: non_neg_integer() | undefined, % = 2, optional, 32 bits
-         content = <<>>         :: iodata() | undefined % = 3, optional
-        }).
--endif.
-
--ifndef('MESSAGE.METRICDATA_PB_H').
--define('MESSAGE.METRICDATA_PB_H', true).
--record('Message.MetricData',
-        {route_key = <<>>       :: iodata() | undefined, % = 1, optional
-         metric = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('MESSAGE_PB_H').
--define('MESSAGE_PB_H', true).
--record('Message',
-        {body                   :: {ping, message_pb:'Message.Ping'()} | {pong, message_pb:'Message.Pong'()} | {pub, message_pb:'Message.Pub'()} | {metric_data, message_pb:'Message.MetricData'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('STREAM.OPEN_PB_H').
--define('STREAM.OPEN_PB_H', true).
--record('Stream.Open',
-        {target = 0             :: non_neg_integer() | undefined % = 1, optional, 32 bits
-        }).
--endif.
-
--ifndef('STREAM.OPENED_PB_H').
--define('STREAM.OPENED_PB_H', true).
--record('Stream.Opened',
-        {
-        }).
--endif.
-
--ifndef('STREAM.OPENERROR_PB_H').
--define('STREAM.OPENERROR_PB_H', true).
--record('Stream.OpenError',
-        {reason = <<>>          :: iodata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('STREAM.DATA_PB_H').
--define('STREAM.DATA_PB_H', true).
--record('Stream.Data',
-        {bytes = <<>>           :: iodata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('STREAM.FIN_PB_H').
--define('STREAM.FIN_PB_H', true).
--record('Stream.Fin',
-        {
-        }).
--endif.
-
--ifndef('STREAM.RESET_PB_H').
--define('STREAM.RESET_PB_H', true).
--record('Stream.Reset',
-        {reason = <<>>          :: iodata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('STREAM_PB_H').
--define('STREAM_PB_H', true).
--record('Stream',
-        {stream_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 32 bits
-         payload                :: {open, message_pb:'Stream.Open'()} | {opened, message_pb:'Stream.Opened'()} | {open_error, message_pb:'Stream.OpenError'()} | {data, message_pb:'Stream.Data'()} | {fin, message_pb:'Stream.Fin'()} | {reset, message_pb:'Stream.Reset'()} | undefined % oneof
-        }).
--endif.
-
--endif.

apps/efka/include/service_pb.hrl

@@ -1,72 +0,0 @@
-%% -*- coding: utf-8 -*-
-%% Automatically generated, do not edit
-%% Generated by gpb_compile version 4.21.7
-
--ifndef(service_pb).
--define(service_pb, true).
-
--define(service_pb_gpb_version, "4.21.7").
-
-
--ifndef('SERVICEREQUEST.REGISTER_PB_H').
--define('SERVICEREQUEST.REGISTER_PB_H', true).
--record('ServiceRequest.Register',
-        {service_id = <<>>      :: unicode:chardata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('SERVICEREQUEST.SUBSCRIBE_PB_H').
--define('SERVICEREQUEST.SUBSCRIBE_PB_H', true).
--record('ServiceRequest.Subscribe',
-        {topic = <<>>           :: unicode:chardata() | undefined % = 1, optional
-        }).
--endif.
-
--ifndef('SERVICEREQUEST_PB_H').
--define('SERVICEREQUEST_PB_H', true).
--record('ServiceRequest',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         request                :: {register, service_pb:'ServiceRequest.Register'()} | {subscribe, service_pb:'ServiceRequest.Subscribe'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('SERVICEREPLY.ERROR_PB_H').
--define('SERVICEREPLY.ERROR_PB_H', true).
--record('ServiceReply.Error',
-        {code = 0               :: integer() | undefined, % = 1, optional, 32 bits
-         message = <<>>         :: unicode:chardata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('SERVICEREPLY_PB_H').
--define('SERVICEREPLY_PB_H', true).
--record('ServiceReply',
-        {packet_id = 0          :: non_neg_integer() | undefined, % = 1, optional, 64 bits
-         reply                  :: {result, iodata()} | {error, service_pb:'ServiceReply.Error'()} | undefined % oneof
-        }).
--endif.
-
--ifndef('SERVICECAST.METRICDATA_PB_H').
--define('SERVICECAST.METRICDATA_PB_H', true).
--record('ServiceCast.MetricData',
-        {route_key = <<>>       :: iodata() | undefined, % = 1, optional
-         metric = <<>>          :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('SERVICECAST.TOPICEVENT_PB_H').
--define('SERVICECAST.TOPICEVENT_PB_H', true).
--record('ServiceCast.TopicEvent',
-        {topic = <<>>           :: unicode:chardata() | undefined, % = 1, optional
-         content = <<>>         :: iodata() | undefined % = 2, optional
-        }).
--endif.
-
--ifndef('SERVICECAST_PB_H').
--define('SERVICECAST_PB_H', true).
--record('ServiceCast',
-        {body                   :: {topic_event, service_pb:'ServiceCast.TopicEvent'()} | {metric_data, service_pb:'ServiceCast.MetricData'()} | undefined % oneof
-        }).
--endif.
-
--endif.

apps/efka/proto/message.proto

@@ -1,179 +0,0 @@
-syntax = "proto3";
-
-// iot <-> efka protocol payload definitions.
-//
-// The transport frame keeps the first byte as the coarse message class:
-//
-//   [CLASS_REQUEST][protobuf(Request)]
-//   [CLASS_RESPONSE][protobuf(Response)]
-//   [CLASS_COMMAND][protobuf(Command)]
-//   [CLASS_COMMAND_RESPONSE][protobuf(CommandResponse)]
-//   [CLASS_MESSAGE][protobuf(Message)]
-//   [CLASS_STREAM][protobuf(Stream)]
-//
-// HTTP proxy bytes carried by Stream.Data are transparent payload bytes.
-
-message Request {
-  uint64 packet_id = 1;
-
-  message AuthRequest {
-    bytes uuid = 1;
-    bytes token = 2;
-    uint64 timestamp = 3;
-  }
-
-  oneof body {
-    AuthRequest auth_request = 10;
-  }
-
-}
-
-message Response {
-  uint64 packet_id = 1;
-
-  message Error {
-    uint32 code = 1;
-    bytes reason = 2;
-  }
-
-  message AuthResponse {
-
-  }
-
-  oneof body {
-    AuthResponse auth_response = 10;
-    Error error = 11;
-  }
-
-}
-
-message Command {
-  uint64 packet_id = 1;
-
-  message Container {
-    message ContainerTarget {
-      bytes name = 1;
-      bytes id = 2;
-    }
-
-    message ContainerList {
-
-    }
-
-    message ContainerStart {
-      ContainerTarget target = 1;
-    }
-
-    message ContainerStop {
-      ContainerTarget target = 1;
-      uint32 timeout_seconds = 2;
-    }
-
-    message ContainerKill {
-      ContainerTarget target = 1;
-      bytes signal = 2;
-    }
-
-    message ContainerRemove {
-      ContainerTarget target = 1;
-      bool force = 2;
-      bool remove_volumes = 3;
-    }
-
-    message ContainerConfig {
-      ContainerTarget target = 1;
-      bytes config = 2;
-    }
-
-    oneof action {
-      ContainerList list = 1;
-      ContainerStart start = 3;
-      ContainerStop stop = 4;
-      ContainerKill kill = 5;
-      ContainerRemove remove = 6;
-      ContainerConfig config = 7;
-    }
-
-  }
-
-  oneof body {
-    Container container = 10;
-  }
-
-}
-
-message CommandResponse {
-  uint64 packet_id = 1;
-
-  message Error {
-    uint32 code = 1;
-    bytes reason = 2;
-  }
-
-  oneof body {
-    bytes result = 10;
-    Error error = 11;
-  }
-
-}
-
-message Message {
-  message Ping {
-  }
-
-  message Pong {
-  }
-
-  message Pub {
-    bytes topic = 1;
-    uint32 qos = 2;
-    bytes content = 3;
-  }
-
-  message MetricData {
-    bytes route_key = 1;
-    bytes metric = 2;
-  }
-
-  oneof body {
-    Ping ping = 10;
-    Pong pong = 11;
-    Pub pub = 12;
-    MetricData metric_data = 13;
-  }
-}
-
-message Stream {
-  uint32 stream_id = 1;
-
-  message Open {
-    uint32 target = 1;
-  }
-
-  message Opened {
-  }
-
-  message OpenError {
-    bytes reason = 1;
-  }
-
-  message Data {
-    bytes bytes = 1;
-  }
-
-  message Fin {
-  }
-
-  message Reset {
-    bytes reason = 1;
-  }
-
-  oneof payload {
-    Open open = 10;
-    Opened opened = 11;
-    OpenError open_error = 12;
-    Data data = 13;
-    Fin fin = 14;
-    Reset reset = 15;
-  }
-}

apps/efka/proto/service.proto

@@ -1,54 +0,0 @@
-syntax = "proto3";
-
-// 微服务通过 ws_channel 与 efka 之间的通道消息定义。
-// 该协议用于替代当前基于 JSON 的请求/响应/推送格式。
-
-message ServiceRequest {
-  uint64 packet_id = 1;
-
-  message Register {
-    string service_id = 1;
-  }
-
-  message Subscribe {
-    string topic = 1;
-  }
-
-  oneof request {
-    Register register = 10;
-    Subscribe subscribe = 11;
-  }
-}
-
-message ServiceReply {
-  uint64 packet_id = 1;
-
-  message Error {
-    int32 code = 1;
-    string message = 2;
-  }
-
-  oneof reply {
-    bytes result = 10;
-    Error error = 11;
-  }
-}
-
-message ServiceCast {
-
-  message MetricData {
-    bytes route_key = 1;
-    bytes metric = 2;
-  }
-
-  message TopicEvent {
-    string topic = 1;
-    bytes content = 2;
-  }
-
-  oneof body {
-    TopicEvent topic_event = 10;
-    MetricData metric_data = 11;
-  }
-
-}

apps/efka/rebar.config

@@ -1,23 +0,0 @@
-{erl_opts, [{i, "include"}]}.
-
-{gpb_opts, [
-    {i, "proto"},
-    {f, ["service.proto", "message.proto"]},
-    recursive,
-    {module_name_prefix, ""},
-    {module_name_suffix, "_pb"},
-    {o_erl, "src/protobuf"},
-    {o_hrl, "include"},
-    include_as_lib,
-    {strings_as_binaries, true},
-    type_specs,
-    report,
-    verbose
-]}.
-
-{provider_hooks, [
-    {pre, [
-        {compile, {protobuf, compile}},
-        {clean, {protobuf, clean}}
-    ]}
-]}.

apps/efka/src/channel/upload_channel.erl

@@ -0,0 +1,79 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 12. 11月 2025 17:08
+%%%-------------------------------------------------------------------
+-module(upload_channel).
+-author("anlicheng").
+
+%% API
+-export([init/2]).
+
+init(Req0, Opts) ->
+    Method = binary_to_list(cowboy_req:method(Req0)),
+    lager:debug("[upload_channel] method is: ~p", [Method]),
+
+    Headers = cowboy_req:headers(Req0),
+    lager:debug("headers is: ~p", [Headers]),
+    case maps:find(<<"content-type">>, Headers) of
+        {ok, <<"application/octet-stream">>} ->
+            Filename = maps:get(<<"x-filename">>, Headers),
+            case filename:extension(Filename) of
+                <<>> ->
+                    Req = cowboy_req:reply(400, #{
+                        <<"Content-Type">> => <<"text/html;charset=utf-8">>
+                    }, <<"Miss file extension">>, Req0),
+                    {ok, Req, Opts};
+                _ ->
+                    Basename = filename:basename(Filename),
+                    handle_raw_file(Req0, binary_to_list(Basename)),
+                    Req2 = cowboy_req:reply(400, #{
+                        <<"Content-Type">> => <<"text/html;charset=utf-8">>
+                    }, <<"ok">>, Req0),
+                    {ok, Req2, Opts}
+            end;
+        {ok, ContentType} ->
+            lager:debug("[upload_channel] unexpect content-type: ~p", [ContentType]),
+            Req = cowboy_req:reply(400, #{
+                <<"Content-Type">> => <<"text/html;charset=utf-8">>
+            }, <<"Expected application/octet-stream">>, Req0),
+            {ok, Req, Opts};
+        error ->
+            Req = cowboy_req:reply(400, #{
+                <<"Content-Type">> => <<"text/html;charset=utf-8">>
+            }, <<"Miss content-type header">>, Req0),
+            {ok, Req, Opts}
+    end.
+
+%% 读取请求体
+handle_raw_file(Req, Basename) ->
+    Filename = make_file(Basename),
+    {ok, IoDevice} = file:open(Filename, [write]),
+    ok = handle_raw_file0(Req, IoDevice),
+    ok = file:close(IoDevice).
+
+handle_raw_file0(Req, IoDevice) ->
+    case cowboy_req:read_body(Req) of
+        {ok, Data, Req1} ->
+            file:write(IoDevice, Data),
+            ok;
+        {more, Data, Req1} ->
+            file:write(IoDevice, Data),
+            handle_raw_file0(Req1, IoDevice)
+    end.
+
+make_file(Basename) when is_list(Basename) ->
+    {ok, UploadDir} = application:get_env(efka, upload_dir),
+    {{Y, M, D}, _} = calendar:local_time(),
+    DateDir = io_lib:format("~p-~p-~p", [Y, M, D]),
+    BaseDir = UploadDir ++ DateDir,
+    case filelib:is_dir(BaseDir) of
+        true ->
+            ok;
+        false ->
+            ok = file:make_dir(BaseDir)
+    end,
+    BaseDir ++ "/" ++ Basename.

apps/efka/src/channel/ws_channel.erl

@@ -0,0 +1,257 @@
+%%%-------------------------------------------------------------------
+%%% @author licheng5
+%%% @copyright (C) 2021, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 11. 1月 2021 上午12:17
+%%%-------------------------------------------------------------------
+-module(ws_channel).
+-author("licheng5").
+-include("efka_tables.hrl").
+
+%% API
+-export([init/2]).
+-export([websocket_init/1, websocket_handle/2, websocket_info/2, terminate/3]).
+
+%% 最大的等待时间
+-define(PENDING_TIMEOUT, 10 * 1000).
+
+-record(state, {
+    service_id :: undefined | binary(),
+    service_pid :: undefined | pid(),
+
+    stream_id = 1,
+    %% #{stream_id => {StreamPid, StreamRef}}
+    stream_map = #{},
+
+    is_registered = false :: boolean()
+}).
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% 逻辑处理方法
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+init(Req, Opts) ->
+    {cowboy_websocket, Req, Opts}.
+
+websocket_init(_State) ->
+    lager:debug("[ws_channel] get a new connection"),
+    %% 初始状态为true
+    {ok, #state{}}.
+
+websocket_handle(ping, State) ->
+    {reply, pong, State};
+
+websocket_handle({text, Data}, State) ->
+    Request = jiffy:decode(Data, [return_maps]),
+    lager:debug("[ws_channle] get request: ~p", [Request]),
+    handle_request(Request, State);
+
+websocket_handle(Info, State) ->
+    lager:error("[ws_channel] get a unknown message: ~p, channel will closed", [Info]),
+    {ok, State}.
+
+%% 订阅的消息
+websocket_info({topic_broadcast, Topic, Content}, State = #state{}) ->
+    Req = iolist_to_binary(jiffy:encode(#{
+        <<"method">> => <<"publish">>,
+        <<"params">> => #{<<"topic">> => Topic, <<"content">> => Content}
+    }, [force_utf8])),
+
+    lager:debug("[ws_channel] will publish topic: ~p, message: ~p", [Topic, Req]),
+
+    {reply, {text, Req}, State};
+
+%% service进程关闭
+websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) ->
+    lager:debug("[ws_channel] container_pid: ~p, exited: ~p", [ServicePid, Reason]),
+    {stop, State#state{service_pid = undefined}};
+
+%% stream进程关闭
+websocket_info({'DOWN', _Ref, process, StreamPid, Reason}, State = #state{stream_map = StreamMap}) ->
+    case search_stream_id(StreamPid, StreamMap) of
+        error ->
+            {ok, State};
+        {ok, StreamId} ->
+            case Reason of
+                normal ->
+                    {ok, State#state{stream_map = maps:remove(StreamId, StreamMap)}};
+                _ ->
+                    PushReply = json_push(#{
+                        <<"stream_reply">> => #{
+                            <<"stream_id">> => StreamId,
+                            <<"result">> => <<"task failed">>
+                        }
+                    }),
+                    {reply, {text, PushReply}, State#state{stream_map = maps:remove(StreamId, StreamMap)}}
+            end
+    end;
+
+%% stream任务完成
+websocket_info({stream_reply, StreamPid, Reply}, State = #state{stream_map = StreamMap}) ->
+    case search_stream_id(StreamPid, StreamMap) of
+        error ->
+            {ok, State};
+        {ok, StreamId} ->
+            PushReply = json_push(#{
+                <<"stream_reply">> => #{
+                    <<"stream_id">> => StreamId,
+                    <<"result">> => Reply
+                }
+            }),
+            {reply, {text, PushReply}, State}
+    end;
+
+%% 处理关闭信号
+websocket_info({stop, Reason}, State) ->
+    lager:debug("[ws_channel] the channel will be closed with reason: ~p", [Reason]),
+    {stop, State};
+
+%% 处理其他未知消息
+websocket_info(Info, State) ->
+    lager:debug("[ws_channel] channel get unknown info: ~p", [Info]),
+    {ok, State}.
+
+%% 进程关闭事件
+terminate(Reason, _Req, State = #state{service_id = ServiceId, is_registered = IsRegistered}) ->
+    case IsRegistered of
+        true ->
+            ok = service_model:change_status(ServiceId, 0);
+        false ->
+            ok
+    end,
+    lager:debug("[ws_channel] channel close with reason: ~p, state is: ~p", [Reason, State]),
+    ok.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% helper methods
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+%% 注册, 要建立程序和容器之间的关系
+handle_request(#{<<"id">> := Id, <<"method">> := <<"register">>, <<"params">> := Params = #{<<"service_id">> := ServiceId}}, State) ->
+    {ok, ServicePid} = efka_service_sup:start_service(ServiceId),
+    case efka_service:attach_channel(ServicePid, self()) of
+        ok ->
+            Reply = json_result(Id, <<"ok">>),
+            erlang:monitor(process, ServicePid),
+
+            %% 更新微服务的状态
+            MetaData = maps:get(<<"meta_data">>, Params, #{}),
+            ContainerName = maps:get(<<"container_name">>, Params, <<>>),
+            ok = service_model:insert(#service{
+                service_id = ServiceId,
+                container_name = ContainerName,
+                status = ?SERVICE_RUNNING,
+                meta_data = MetaData,
+                create_ts = efka_util:timestamp(),
+                update_ts = efka_util:timestamp()
+            }),
+
+            {reply, {text, Reply}, State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}};
+        {error, Error} ->
+            lager:warning("[ws_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]),
+            {stop, State}
+    end;
+
+%% 订阅事件
+handle_request(#{<<"id">> := Id, <<"method">> := <<"subscribe">>, <<"params">> := #{<<"topic">> := Topic}}, State = #state{is_registered = true}) ->
+    Reply = case efka_subscription:subscribe(Topic, self()) of
+                ok ->
+                    json_result(Id, <<"ok">>);
+                {error, Reason} ->
+                    json_error(Id, -1, Reason)
+            end,
+    {reply, {text, Reply}, State};
+
+%% 文件上传
+handle_request(#{<<"id">> := Id, <<"method">> := <<"new_stream">>,
+    <<"params">> := #{<<"file_name">> := Filename0, <<"file_size">> := FileSize}}, State = #state{stream_id = StreamId, stream_map = StreamMap, is_registered = true}) ->
+    Filename = filename:basename(binary_to_list(Filename0)),
+
+    {ok, {StreamPid, StreamRef}} = efka_stream:start_monitor(self()),
+    {ok, Path} = efka_stream:setup(StreamPid, Filename, FileSize),
+
+    Reply = json_result(Id, #{
+        <<"stream_id">> => StreamId,
+        <<"path">> => Path
+    }),
+    {reply, {text, Reply}, State#state{stream_id = StreamId + 1, stream_map = maps:put(StreamId, {StreamPid, StreamRef}, StreamMap)}};
+
+handle_request(#{<<"method">> := <<"stream_chunk">>,
+    <<"params">> := #{<<"stream_id">> := StreamId, <<"chunk_data">> := ChunkData}}, State = #state{stream_map = StreamMap, is_registered = true}) ->
+    case maps:find(StreamId, StreamMap) of
+        error ->
+            {ok, State};
+        {ok, {StreamPid, _}} ->
+            case ChunkData =:= <<>> of
+                true ->
+                    efka_stream:finish(StreamPid);
+                false ->
+                    efka_stream:data(StreamPid, ChunkData)
+            end,
+            {ok, State}
+    end;
+
+%% 数据项
+handle_request(#{<<"method">> := <<"metric_data">>,
+    <<"params">> := #{<<"route_key">> := RouteKey, <<"metric">> := Metric0}}, State = #state{service_pid = ServicePid, is_registered = true}) ->
+    case map_metric(Metric0) of
+        {ok, Metric} ->
+            efka_service:metric_data(ServicePid, RouteKey, Metric);
+        error ->
+            lager:debug("[ws_channel] metric_data get invalid metric: ~p", Metric0)
+    end,
+    {ok, State}.
+
+-spec json_result(Id :: integer(), Result :: term()) -> binary().
+json_result(Id, Result) when is_integer(Id) ->
+    Response = #{
+        <<"id">> => Id,
+        <<"result">> => Result
+    },
+    jiffy:encode(Response, [force_utf8]).
+
+-spec json_error(Id :: integer(), Code :: integer(), Message :: binary()) -> binary().
+json_error(Id, Code, Message) when is_integer(Id), is_integer(Code), is_binary(Message) ->
+    Response = #{
+        <<"id">> => Id,
+        <<"error">> => #{<<"code">> => Code, <<"message">> => Message}
+    },
+    jiffy:encode(Response, [force_utf8]).
+
+-spec json_push(Result :: term()) -> binary().
+json_push(Result) ->
+    Response = #{
+        <<"push">> => Result
+    },
+    jiffy:encode(Response, [force_utf8]).
+
+-spec search_stream_id(StreamPid :: pid(), StreamMap :: map()) -> error | {ok, StreamId :: integer()}.
+search_stream_id(StreamPid, StreamMap) when is_pid(StreamPid), is_map(StreamMap) ->
+    StreamIds = lists:filtermap(fun({StreamId, {StreamPid0, _}}) ->
+        case StreamPid0 =:= StreamPid of
+            true ->
+                {true, StreamId};
+            false ->
+                false
+        end
+                                end, maps:to_list(StreamMap)),
+    case StreamIds of
+        [] ->
+            error;
+        [StreamId|_] ->
+            {ok, StreamId}
+    end.
+
+-spec map_metric(Metric :: any()) -> {ok, binary()} | error.
+map_metric(Metric) when is_binary(Metric) ->
+    {ok, Metric};
+map_metric(Metric) when is_map(Metric) orelse is_list(Metric) ->
+    {ok, jiffy:encode(Metric, [force_utf8])};
+map_metric(Metric) when is_integer(Metric) ->
+    {ok, integer_to_binary(Metric)};
+map_metric(Metric) when is_float(Metric) ->
+    {ok, erlang:float_to_binary(Metric, [compact, {decimals, 10}])};
+map_metric(_) ->
+    error.

apps/efka/src/docker/docker_commands.erl

@@ -0,0 +1,466 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 15. 9月 2025 16:11
+%%%-------------------------------------------------------------------
+-module(docker_commands).
+-author("anlicheng").
+
+%% API
+-export([pull_image/2, check_image_exist/1]).
+-export([create_container/3, check_container_exist/1, is_container_running/1,
+    start_container/1, stop_container/1, remove_container/1, kill_container/1,
+    get_containers/0]).
+
+-spec pull_image(Image :: binary(), Callback :: fun((Msg :: any()) -> no_return())) -> ok | {error, Reason :: any()}.
+pull_image(Image, Callback) when is_binary(Image), is_function(Callback, 1) ->
+    Url = lists:flatten(io_lib:format("/images/create?fromImage=~s", [binary_to_list(Image)])),
+    docker_http:stream_request(Callback, "POST", Url, <<>>, []).
+
+-spec check_image_exist(Image :: binary()) -> boolean().
+check_image_exist(Image) when is_binary(Image) ->
+    Url = lists:flatten(io_lib:format("/images/~s/json", [binary_to_list(Image)])),
+    case docker_http:request("GET", Url, <<"">>, []) of
+        {ok, 200, _Headers, Resp} ->
+            M = catch jiffy:decode(Resp, [return_maps]),
+            is_map(M) andalso maps:is_key(<<"Id">>, M);
+        _ ->
+            false
+    end.
+
+-spec create_container(ContainerName :: binary(), ContainerDir :: string(), Config :: map()) -> {ok, ContainerId :: binary()} | {error, Reason :: any()}.
+create_container(ContainerName, ContainerDir, Config) when is_binary(ContainerName), is_list(ContainerDir), is_map(Config) ->
+    Url = lists:flatten(io_lib:format("/containers/create?name=~s", [binary_to_list(ContainerName)])),
+    %% 挂载预留的目录，用来作为配置文件的存放
+    ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)),
+
+    %% 增加自定义的用来放配置文件的目录
+    Volumes0 = maps:get(<<"volumes">>, Config, []),
+    Volumes = [<<ConfigFile/binary, ":/usr/local/etc/service.conf">>|Volumes0],
+    NewConfig = Config#{<<"volumes">> => Volumes},
+
+    Options = build_options(ContainerName, NewConfig),
+    display_options(Options),
+
+    Body = iolist_to_binary(jiffy:encode(Options, [force_utf8])),
+    true = is_binary(Body),
+
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_http:request("POST", Url, Body, Headers) of
+        {ok, 201, _Headers, Resp} ->
+            case catch jiffy:decode(Resp, [return_maps]) of
+                #{<<"Id">> := ContainerId} when is_binary(ContainerId) ->
+                    {ok, ContainerId};
+                _ ->
+                    {error, Resp}
+            end;
+        {ok, _StatusCode, _, ErrorResp} ->
+            case catch jiffy:decode(ErrorResp, [return_maps]) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec is_container_running(ContainerId :: binary()) -> boolean().
+is_container_running(ContainerId) when is_binary(ContainerId) ->
+    case inspect_container(ContainerId) of
+        {ok, #{<<"State">> := #{<<"Running">> := Running}}} ->
+            Running;
+        {error, _} ->
+            false
+    end.
+
+-spec check_container_exist(ContainerName :: binary()) -> boolean().
+check_container_exist(ContainerName) when is_binary(ContainerName) ->
+    case inspect_container(ContainerName) of
+        {ok, #{<<"Id">> := Id}} when is_binary(Id) ->
+            true;
+        _ ->
+            false
+    end.
+
+-spec start_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+start_container(ContainerName) when is_binary(ContainerName) ->
+    Url = lists:flatten(io_lib:format("/containers/~s/start", [binary_to_list(ContainerName)])),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_http:request("POST", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, 304, _Headers, _} ->
+            {error, <<"container already started">>};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch jiffy:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec stop_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+stop_container(ContainerName) when is_binary(ContainerName) ->
+    Url = lists:flatten(io_lib:format("/containers/~s/stop", [binary_to_list(ContainerName)])),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_http:request("POST", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, 304, _Headers, _} ->
+            {error, <<"container already stopped">>};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch jiffy:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec kill_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+kill_container(ContainerName) when is_binary(ContainerName) ->
+    Url = lists:flatten(io_lib:format("/containers/~s/kill", [binary_to_list(ContainerName)])),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_http:request("POST", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch jiffy:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec remove_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}.
+remove_container(ContainerName) when is_binary(ContainerName) ->
+    Url = lists:flatten(io_lib:format("/containers/~s", [binary_to_list(ContainerName)])),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_http:request("DELETE", Url, <<>>, Headers) of
+        {ok, 204, _Headers, _} ->
+            ok;
+        {ok, 304, _Headers, _} ->
+            {error, <<"container already stopped">>};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch jiffy:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec get_containers() -> {ok, Containers :: [map()]} | {error, Reason :: binary()}.
+get_containers() ->
+    Url = "/containers/json?all=true",
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_http:request("GET", Url, <<>>, Headers) of
+        {ok, 200, _Headers, ContainersBin} ->
+            Containers = jiffy:decode(ContainersBin, [return_maps]),
+            {ok, Containers};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch jiffy:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+-spec inspect_container(ContainerId :: binary()) -> {ok, Json :: map()} | {error, Error :: any()}.
+inspect_container(ContainerId) when is_binary(ContainerId) ->
+    Url = lists:flatten(io_lib:format("/containers/~s/json", [binary_to_list(ContainerId)])),
+    Headers = [
+        {<<"Content-Type">>, <<"application/json">>}
+    ],
+    case docker_http:request("GET", Url, <<>>, Headers) of
+        {ok, 200, _Headers, Resp} ->
+            Json = jiffy:decode(Resp, [return_maps]),
+            {ok, Json};
+        {ok, _StatusCode, _Header, ErrorResp} ->
+            case catch jiffy:decode(ErrorResp) of
+                #{<<"message">> := Msg} ->
+                    {error, Msg};
+                _ ->
+                    {error, ErrorResp}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%% helper methods
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+%% 构建最终 JSON Map
+build_options(ContainerName, Config) when is_binary(ContainerName), is_map(Config) ->
+    %% 容器名称作为环境变量，注入到运行时环境中
+    Envs0 = maps:get(<<"envs">>, Config, []),
+    Envs = [<<"CONTAINER_NAME=", ContainerName/binary>>|Envs0],
+    #{
+        <<"Image">> => maps:get(<<"image">>, Config, <<>>),
+        <<"Cmd">> => maps:get(<<"command">>, Config, []),
+        <<"Entrypoint">> => maps:get(<<"entrypoint">>, Config, []),
+        <<"Env">> => Envs,
+        <<"Labels">> => maps:get(<<"labels">>, Config, #{}),
+        <<"Volumes">> => build_volumes(Config),
+        <<"User">> => maps:get(<<"user">>, Config, <<>>),
+        <<"WorkingDir">> => maps:get(<<"working_dir">>, Config, <<>>),
+        <<"Hostname">> => maps:get(<<"hostname">>, Config, <<>>),
+        <<"ExposedPorts">> => build_expose(Config),
+        <<"NetworkingConfig">> => build_networks(Config),
+        <<"Healthcheck">> => build_healthcheck(Config),
+        <<"HostConfig">> => fold_merge([
+            build_network_mode(Config),
+            build_binds(Config),
+            build_restart(Config),
+            build_privileged(Config),
+            build_cap_add_drop(Config),
+            build_devices(Config),
+            build_memory(Config),
+            build_cpu(Config),
+            build_ulimits(Config),
+            build_tmpfs(Config),
+            build_sysctls(Config),
+            build_extra_hosts(Config)
+        ])
+    }.
+
+%% 工具函数
+fold_merge(List) ->
+    lists:foldl(fun maps:merge/2, #{}, List).
+
+%% --- 构建子字段 ---
+build_expose(Config) ->
+    Ports = maps:get(<<"expose">>, Config, []),
+    case Ports of
+        [] -> #{};
+        _ ->
+            maps:from_list([{<<P/binary, "/tcp">>, #{}} || P <- Ports])
+    end.
+
+build_volumes(Config) ->
+    Vols = maps:get(<<"volumes">>, Config, []),
+    case Vols of
+        [] ->
+            #{};
+        _ ->
+            maps:from_list(lists:map(fun(V) ->
+                [_Host, Cont] = binary:split(V, <<":">>, []),
+                {Cont, #{}}
+                                     end, Vols))
+    end.
+
+build_binds(Config) ->
+    Vols = maps:get(<<"volumes">>, Config, []),
+    case Vols of
+        [] ->
+            #{};
+        _ ->
+            #{<<"Binds">> => Vols}
+    end.
+
+build_networks(Config) ->
+    Nets = maps:get(<<"networks">>, Config, []),
+    case Nets of
+        [] -> #{};
+        _ ->
+            NetCfg = maps:from_list([{N, #{}} || N <- Nets]),
+            #{<<"EndpointsConfig">> => NetCfg}
+    end.
+
+build_network_mode(Config) ->
+    NetworkMode = maps:get(<<"network_mode">>, Config, <<"bridge">>),
+    #{<<"NetworkMode">> => NetworkMode}.
+
+parse_mem(Val) ->
+    case binary:last(Val) of
+        $m ->
+            N = binary:part(Val, {0, byte_size(Val)-1}),
+            list_to_integer(binary_to_list(N)) * 1024 * 1024;
+        $g ->
+            N = binary:part(Val, {0, byte_size(Val)-1}),
+            list_to_integer(binary_to_list(N)) * 1024 * 1024 * 1024;
+        _ ->
+            list_to_integer(binary_to_list(Val))
+    end.
+
+build_healthcheck(Config) ->
+    HC = maps:get(<<"healthcheck">>, Config, #{}),
+    case maps:size(HC) of
+        0 ->
+            #{};
+        _ ->
+            #{
+                <<"Test">> => maps:get(<<"test">>, HC, []),
+                <<"Interval">> => parse_duration(maps:get(<<"interval">>, HC, <<"0s">>)),
+                <<"Timeout">> => parse_duration(maps:get(<<"timeout">>, HC, <<"0s">>)),
+                <<"Retries">> => maps:get(<<"retries">>, HC, 0)
+            }
+    end.
+
+parse_duration(Bin) ->
+    %% "30s" -> 30000000000
+    Sz = byte_size(Bin),
+    NBin = binary:part(Bin, {0, Sz-1}),
+    N = list_to_integer(binary_to_list(NBin)),
+    case binary:last(Bin) of
+        $s ->
+            N * 1000000000;
+        $m ->
+            N * 60000000000;
+        _ ->
+            N
+    end.
+
+%% --- 构建子字段 ---
+
+build_restart(Config) ->
+    case maps:get(<<"restart">>, Config, undefined) of
+        undefined ->
+            #{};
+        Policy ->
+            #{<<"RestartPolicy">> => #{<<"Name">> => Policy}}
+    end.
+
+build_privileged(Config) ->
+    case maps:get(<<"privileged">>, Config, false) of
+        true ->
+            #{<<"Privileged">> => true};
+        _ ->
+            #{}
+    end.
+
+build_cap_add_drop(Config) ->
+    Add = maps:get(<<"cap_add">>, Config, []),
+    Drop = maps:get(<<"cap_drop">>, Config, []),
+    case {Add, Drop} of
+        {[], []} ->
+            #{};
+        _ ->
+            #{<<"CapAdd">> => Add, <<"CapDrop">> => Drop}
+    end.
+
+build_devices(Config) ->
+    Devs = maps:get(<<"devices">>, Config, []),
+    case Devs of
+        [] ->
+            #{};
+        _ ->
+            DevObjs = [#{<<"PathOnHost">> => H, <<"PathInContainer">> => C,
+                <<"CgroupPermissions">> => <<"rwm">>}
+                || D <- Devs,
+                [H, C] <- [binary:split(D, <<":">>, [])]],
+            #{<<"Devices">> => DevObjs}
+    end.
+
+build_memory(Config) ->
+    Mem = maps:get(<<"mem_limit">>, Config, undefined),
+    MemRes = maps:get(<<"mem_reservation">>, Config, undefined),
+    HCfg = #{},
+    HCfg1 = if
+                Mem /= undefined ->
+                    maps:put(<<"Memory">>, parse_mem(Mem), HCfg);
+                true ->
+                    HCfg
+            end,
+    if
+        MemRes /= undefined ->
+            maps:put(<<"MemoryReservation">>, parse_mem(MemRes), HCfg1);
+        true ->
+            HCfg1
+    end.
+
+
+build_cpu(Config) ->
+    CPU = maps:get(<<"cpus">>, Config, undefined),
+    Shares = maps:get(<<"cpu_shares">>, Config, undefined),
+    HCfg = #{},
+    HCfg1 = if
+                CPU /= undefined ->
+                    maps:put(<<"NanoCpus">>, trunc(CPU * 1000000000), HCfg);
+                true ->
+                    HCfg
+            end,
+    if
+        Shares /= undefined ->
+            maps:put(<<"CpuShares">>, Shares, HCfg1);
+        true ->
+            HCfg1
+    end.
+
+build_ulimits(Config) ->
+    UL = maps:get(<<"ulimits">>, Config, #{}),
+    case maps:size(UL) of
+        0 ->
+            #{};
+        _ ->
+            ULList = lists:map(fun({K, V}) ->
+                [S1, H1] = binary:split(V, <<":">>, []),
+                S = list_to_integer(binary_to_list(S1)),
+                H = list_to_integer(binary_to_list(H1)),
+                #{<<"Name">> => K, <<"Soft">> => S, <<"Hard">> => H}
+                               end, maps:to_list(UL)),
+
+            #{<<"Ulimits">> => ULList}
+    end.
+
+build_sysctls(Config) ->
+    SC = maps:get(<<"sysctls">>, Config, #{}),
+    case maps:size(SC) of
+        0 ->
+            #{};
+        _ ->
+            #{<<"Sysctls">> => SC}
+    end.
+
+build_tmpfs(Config) ->
+    Tmp = maps:get(<<"tmpfs">>, Config, []),
+    case Tmp of
+        [] ->
+            #{};
+        _ ->
+            #{<<"Tmpfs">> => maps:from_list([{T, <<>>} || T <- Tmp])}
+    end.
+
+build_extra_hosts(Config) ->
+    Hosts = maps:get(<<"extra_hosts">>, Config, []),
+    case Hosts of
+        [] ->
+            #{};
+        _ ->
+            #{<<"ExtraHosts">> => Hosts}
+    end.
+
+-spec display_options(Options :: map()) -> no_return().
+display_options(Options) when is_map(Options) ->
+    lager:debug("deploy options: ~p", [jiffy:encode(Options, [force_utf8])]),
+    lists:foreach(fun({K, V}) -> lager:debug("~p => ~p", [K, V]) end, maps:to_list(Options)).

apps/efka/src/docker/docker_deployer.erl

@@ -0,0 +1,117 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 07. 5月 2025 15:47
+%%%-------------------------------------------------------------------
+-module(docker_deployer).
+-author("anlicheng").
+-dialyzer([{nowarn_function, normalize_image/1}]).
+
+%% API
+-export([start_monitor/3]).
+-export([deploy/3]).
+
+-define(TASK_SUCCESS, <<"success">>).
+-define(TASK_FAIL, <<"fail">>).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_monitor(TaskId :: integer(), ContainerDir :: string(), Config :: map()) -> {ok, {Pid :: pid(), MRef :: reference()}}).
+start_monitor(TaskId, ContainerDir, Config) when is_integer(TaskId), is_list(ContainerDir), is_map(Config) ->
+    {ok, spawn_monitor(?MODULE, deploy, [TaskId, ContainerDir, Config])}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+%{
+%   "image": "nginx:latest",
+%   "container_name": "my_nginx",
+%   "ports": ["8080:80", "443:443"],
+%   "volumes": ["/host/data:/data", "/host/log:/var/log"],
+%   "envs": ["ENV1=val1", "ENV2=val2"],
+%   "entrypoint": ["/docker-entrypoint.sh"],
+%   "command": ["nginx", "-g", "daemon off;"],
+%   "restart": "always"
+%}
+-spec deploy(TaskId :: integer(), ContainerDir :: string(), Config :: map()) -> no_return().
+deploy(TaskId, ContainerDir, Config) when is_integer(TaskId), is_list(ContainerDir), is_map(Config) ->
+    %% 尝试拉取镜像
+    ContainerName = maps:get(<<"container_name">>, Config),
+    trace_log(TaskId, <<"info">>, <<"开始部署容器:"/utf8, ContainerName/binary>>),
+
+    case docker_commands:check_container_exist(ContainerName) of
+        true ->
+            trace_log(TaskId, <<"info">>, <<"本地容器已经存在:"/utf8, ContainerName/binary>>),
+            efka_remote_agent:close_task_event_stream(TaskId, ?TASK_FAIL);
+        false ->
+            Image0 = maps:get(<<"image">>, Config),
+            Image = normalize_image(Image0),
+
+            trace_log(TaskId, <<"info">>, <<"使用镜像:"/utf8, Image/binary>>),
+            PullResult = case docker_commands:check_image_exist(Image) of
+                             true ->
+                                 trace_log(TaskId, <<"info">>, <<"镜像本地已存在:"/utf8, Image/binary>>),
+                                 ok;
+                             false ->
+                                 trace_log(TaskId, <<"info">>, <<"开始拉取镜像:"/utf8, Image/binary>>),
+                                 CB = fun
+                                          ({message, M}) ->
+                                              trace_log(TaskId, <<"info">>, M);
+                                          ({error, Error}) ->
+                                              trace_log(TaskId, <<"error">>, Error)
+                                      end,
+                                 docker_commands:pull_image(Image, CB)
+                         end,
+
+            case PullResult of
+                ok ->
+                    trace_log(TaskId, <<"info">>, <<"开始创建容器: "/utf8, ContainerName/binary>>),
+                    case docker_commands:create_container(ContainerName, ContainerDir, Config) of
+                        {ok, ContainerId} ->
+                            %% 创建容器对应的配置文件
+                            ConfigFile = docker_helper:get_config_file(ContainerDir),
+                            case file:open(ConfigFile, [write, exclusive]) of
+                                {ok, FD} ->
+                                    ok = file:write(FD, <<>>),
+                                    file:close(FD);
+                                {error, Reason} ->
+                                    Reason1 = list_to_binary(io_lib:format("~p", [Reason])),
+                                    trace_log(TaskId, <<"notice">>, <<"创建配置文件失败: "/utf8, Reason1/binary>>)
+                            end,
+                            ShortContainerId = binary:part(ContainerId, 1, 12),
+                            trace_log(TaskId, <<"info">>, <<"容器创建成功: "/utf8, ShortContainerId/binary>>),
+                            trace_log(TaskId, <<"info">>, <<"任务完成"/utf8>>),
+                            efka_remote_agent:close_task_event_stream(TaskId, ?TASK_SUCCESS);
+                        {error, Reason} ->
+                            trace_log(TaskId, <<"error">>, <<"容器创建失败: "/utf8, Reason/binary>>),
+                            trace_log(TaskId, <<"error">>, <<"任务失败"/utf8>>),
+                            efka_remote_agent:close_task_event_stream(TaskId, ?TASK_FAIL)
+                    end;
+                {error, Reason} ->
+                    trace_log(TaskId, <<"error">>, <<"镜像拉取失败: "/utf8, Reason/binary>>),
+                    efka_remote_agent:close_task_event_stream(TaskId, ?TASK_FAIL)
+            end
+    end.
+
+-spec normalize_image(binary()) -> binary().
+normalize_image(Image) when is_binary(Image) ->
+    Parts = binary:split(Image, <<"/">>, [global]),
+    {PrefixParts, [Last]} = lists:split(length(Parts) - 1, Parts),
+    NormalizedLast = case binary:split(Last, <<":">>, [global]) of
+                         [_Name] -> <<Last/binary, ":latest">>;
+                         [_Name, _Tag] -> Last
+                     end,
+    iolist_to_binary(lists:join(<<"/">>, PrefixParts ++ [NormalizedLast])).
+
+-spec trace_log(TaskId :: integer(), Level :: binary(), Msg :: binary()) -> no_return().
+trace_log(TaskId, Level, Msg) when is_integer(TaskId), is_binary(Level), is_binary(Msg) ->
+    efka_remote_agent:task_event_stream(TaskId, Level, Msg),
+    Info = iolist_to_binary([<<"task_id=">>, integer_to_binary(TaskId), <<" ">>, Level, <<" ">>, Msg]),
+    efka_logger:write(Info).

apps/efka/src/docker/docker_events.erl

@@ -30,7 +30,7 @@
 %%% API
 %%%===================================================================
 
--spec monitor_container(ReceiverPid :: pid(), ContainerId :: binary()) -> ok.
+-spec monitor_container(ReceiverPid :: pid(), ContainerId :: binary()) -> no_return().
 monitor_container(ReceiverPid, ContainerId) when is_pid(ReceiverPid), is_binary(ContainerId) ->
     gen_server:cast(?SERVER, {monitor_container, ReceiverPid, ContainerId}).
 
@@ -93,8 +93,8 @@ handle_info({timeout, _, attach_docker_events}, State = #state{port = undefined}
             {noreply, State}
     end;
 handle_info({Port, {data, {eol, BinLine}}}, State = #state{port = Port}) ->
-    Event = catch json:decode(BinLine),
-    logger:debug("event: ~p", [Event]),
+    Event = catch jiffy:decode(BinLine, [return_maps]),
+    lager:debug("event: ~p", [Event]),
     handle_event(Event, State),
     {noreply, State};
 
@@ -105,7 +105,7 @@ handle_info({'DOWN', MRef, process, _Pid, _Reason}, State = #state{monitors = Mo
 
 %% Port退出的时候，尝试重启
 handle_info({'EXIT', Port, Reason}, State = #state{port = Port}) ->
-    logger:warning("[efka_docker_events] exit with reason: ~p", [Reason]),
+    lager:warning("[efka_docker_events] exit with reason: ~p", [Reason]),
     try_attach_events(5000),
     {noreply, State#state{port = undefined}}.
 
@@ -130,7 +130,6 @@ code_change(_OldVsn, State = #state{}, _Extra) ->
 %%%===================================================================
 %%% Internal functions
 %%%===================================================================
--spec handle_event(term(), #state{}) -> ok.
 handle_event(#{<<"Type">> := <<"container">>, <<"status">> := Status, <<"id">> := Id}, #state{monitors = Monitors}) ->
     case maps:find(Id, Monitors) of
         error ->
@@ -148,6 +147,5 @@ handle_event(#{<<"Type">> := <<"container">>, <<"status">> := Status, <<"id">> :
 handle_event(_, _) ->
     ok.
 
--spec try_attach_events(non_neg_integer()) -> reference().
 try_attach_events(Timeout) ->
     erlang:start_timer(Timeout, self(), attach_docker_events).

apps/efka/src/docker/docker_helper.erl

@@ -0,0 +1,36 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 17. 9月 2025 14:50
+%%%-------------------------------------------------------------------
+-module(docker_helper).
+-author("anlicheng").
+
+%% API
+-export([ensure_container_dir/2, get_container_dir/2, get_config_file/1]).
+
+-spec ensure_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()}.
+ensure_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
+    %% 根目录
+    ContainerRootDir = RootDir ++ "/" ++ binary_to_list(ContainerName) ++ "/",
+    ok = filelib:ensure_dir(ContainerRootDir),
+    {ok, ContainerRootDir}.
+
+-spec get_config_file(ContainerDir :: string()) -> ConfigFile :: string().
+get_config_file(ContainerDir) when is_list(ContainerDir) ->
+    %% 根目录
+    ContainerDir ++ "service.conf".
+
+-spec get_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()} | error.
+get_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) ->
+    %% 根目录
+    ContainerRootDir = RootDir ++ "/" ++ binary_to_list(ContainerName) ++ "/",
+    case filelib:is_dir(ContainerRootDir) of
+        true ->
+            {ok, ContainerRootDir};
+        false ->
+            error
+    end.

apps/efka/src/docker/docker_http.erl

@@ -0,0 +1,85 @@
+%%% docker_http.erl
+-module(docker_http).
+-export([request/4, stream_request/5]).
+
+%% 通过 Unix Socket 调用 Docker API
+-spec request(Method :: string(), Path :: string(), Body :: binary() | undefined, Headers :: list()) ->
+    {ok, StatusCode :: integer(), RespHeaders :: proplists:proplist(), RespBody :: binary()} | {error, any()}.
+request(Method, Path, Body, Headers) when is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
+    SocketPath = "/var/run/docker.sock",
+    %% 使用 gun:open/2 + {local, Path} 方式
+    case gun:open_unix(SocketPath, #{}) of
+        {ok, ConnPid} ->
+            case gun:await_up(ConnPid) of
+                {ok, _} ->
+                    %% 发送 HTTP 请求
+                    StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
+                    receive_response(ConnPid, StreamRef);
+                {error, Reason} ->
+                    {error, Reason}
+            end;
+        {error, Reason} ->
+            {error, Reason}
+    end.
+
+receive_response(ConnPid, StreamRef) ->
+    receive
+        {gun_response, ConnPid, StreamRef, nofin, Status, Headers} ->
+            receive_body(ConnPid, StreamRef, Status, Headers, <<>>);
+        {gun_response, ConnPid, StreamRef, fin, Status, Headers} ->
+            {ok, Status, Headers, <<>>};
+        {gun_down, ConnPid, _, Reason, _} ->
+            {error, {http_closed, Reason}}
+    after 5000 ->
+        {error, timeout}
+    end.
+receive_body(ConnPid, StreamRef, Status, Headers, Acc) ->
+    receive
+        {gun_data, ConnPid, StreamRef, fin, Data} ->
+            {ok, Status, Headers, <<Acc/binary, Data/binary>>};
+        {gun_data, ConnPid, StreamRef, nofin, Data} ->
+            NewAcc = <<Acc/binary, Data/binary>>,
+            receive_body(ConnPid, StreamRef, Status, Headers, NewAcc)
+    after 10000 ->
+        {error, timeout22}
+    end.
+
+%% 通过 Unix Socket 调用 Docker API
+-spec stream_request(Callback :: any(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) -> ok | {error, Reason :: any()}.
+stream_request(Callback, Method, Path, Body, Headers) when is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) ->
+    SocketPath = "/var/run/docker.sock",
+    case gun:open_unix(SocketPath, #{}) of
+        {ok, ConnPid} ->
+            case gun:await_up(ConnPid) of
+                {ok, _} ->
+                    %% 发送 HTTP 请求
+                    StreamRef = gun:request(ConnPid, Method, Path, Headers, Body),
+                    receive_response(Callback, ConnPid, StreamRef);
+                {error, Reason} ->
+                    {error, Reason}
+            end;
+        {error, Reason} ->
+            Callback({error, Reason}),
+            {error, Reason}
+    end.
+
+receive_response(Callback, ConnPid, StreamRef) ->
+    receive
+        {gun_response, ConnPid, StreamRef, nofin, _Status, _Headers} ->
+            receive_body(Callback, ConnPid, StreamRef);
+        {gun_down, ConnPid, _, Reason, _} ->
+            Callback({error, Reason}),
+            {error, Reason}
+    after 5000 ->
+        Callback({error, <<"处理超时"/utf8>>}),
+        {error, timeout}
+    end.
+receive_body(Callback, ConnPid, StreamRef) ->
+    receive
+        {gun_data, ConnPid, StreamRef, fin, Data} ->
+            Callback({message, Data}),
+            ok;
+        {gun_data, ConnPid, StreamRef, nofin, Data} ->
+            Callback({message, Data}),
+            receive_body(Callback, ConnPid, StreamRef)
+    end.

apps/efka/src/docker/docker_manager.erl

@@ -0,0 +1,225 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%% 微服务守护进程
+%%% 1. 负责微服务的下载, 版本的管理
+%%% 2. 目录管理等
+%%% @end
+%%% Created : 19. 4月 2025 14:55
+%%%-------------------------------------------------------------------
+-module(docker_manager).
+-author("anlicheng").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0]).
+-export([deploy/2, start_container/1, stop_container/1, config_container/2, kill_container/1, remove_container/1]).
+-export([get_containers/0]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+-record(state, {
+    root_dir :: string(),
+    %% 建立任务到ref之间的映射, #{TaskPid => TaskId}
+    task_map = #{}
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec get_containers() -> {ok, Containers :: [map()]} | {error, Reason :: binary()}.
+get_containers() ->
+    gen_server:call(?SERVER, get_containers).
+
+-spec deploy(TaskId :: integer(), Config :: map()) -> ok | {error, Reason :: binary()}.
+deploy(TaskId, Config) when is_integer(TaskId), is_map(Config) ->
+    gen_server:call(?SERVER, {deploy, TaskId, Config}).
+
+-spec config_container(ContainerName :: binary(), Config :: binary()) -> ok | {error, Reason :: binary()}.
+config_container(ContainerName, Config) when is_binary(ContainerName), is_binary(Config) ->
+    gen_server:call(?SERVER, {config_container, ContainerName, Config}).
+
+-spec start_container(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
+start_container(ContainerId) when is_binary(ContainerId) ->
+    gen_server:call(?SERVER, {start_container, ContainerId}).
+
+-spec stop_container(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
+stop_container(ContainerId) when is_binary(ContainerId) ->
+    gen_server:call(?SERVER, {stop_container, ContainerId}).
+
+-spec kill_container(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
+kill_container(ContainerId) when is_binary(ContainerId) ->
+    gen_server:call(?SERVER, {kill_container, ContainerId}).
+
+-spec remove_container(ServiceId :: binary()) -> ok | {error, Reason :: term()}.
+remove_container(ContainerId) when is_binary(ContainerId) ->
+    gen_server:call(?SERVER, {remove_container, ContainerId}).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link() ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([]) ->
+    erlang:process_flag(trap_exit, true),
+    {ok, RootDir} = application:get_env(efka, root_dir),
+    {ok, #state{root_dir = RootDir}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call({deploy, TaskId, Config = #{<<"container_name">> := ContainerName}}, _From, State = #state{root_dir = RootDir, task_map = TaskMap}) ->
+    %% 创建目录
+    {ok, ContainerDir} = docker_helper:ensure_container_dir(RootDir, ContainerName),
+    {ok, {TaskPid, _Ref}} = docker_deployer:start_monitor(TaskId, ContainerDir, Config),
+    lager:debug("[docker_manager] start deploy task_id: ~p, config: ~p", [TaskId, Config]),
+    {reply, ok, State#state{task_map = maps:put(TaskPid, TaskId, TaskMap)}};
+
+%% 处理容器关联的配置文件
+handle_call({config_container, ContainerName, Config}, _From, State = #state{root_dir = RootDir}) ->
+    case docker_helper:get_container_dir(RootDir, ContainerName) of
+        {ok, ContainerDir} ->
+            %% 覆盖容器的配置文件
+            ConfigFile = docker_helper:get_config_file(ContainerDir),
+            case file:write_file(ConfigFile, Config, [write, binary]) of
+                ok ->
+                    lager:warning("[docker_manager] write config file: ~p success", [ConfigFile]),
+                    {reply, ok, State};
+                {error, Reason} ->
+                    lager:warning("[docker_manager] write config file: ~p, get error: ~p", [ConfigFile, Reason]),
+                    {reply, {error, <<"write config failed">>}, State}
+            end;
+        error ->
+            {reply, {error, <<"error">>}, State}
+    end;
+
+%% 启动服务: 当前服务如果正常运行，则不允许重启
+handle_call({start_container, ContainerId}, _From, State) ->
+    case docker_commands:start_container(ContainerId) of
+        ok ->
+            {reply, ok, State};
+        {error, Reason} ->
+            {reply, {error, Reason}, State}
+    end;
+
+
+%% 停止服务, 主动停止的时候会改变服务配置的status字段
+handle_call({stop_container, ContainerId}, _From, State = #state{}) ->
+    case docker_commands:stop_container(ContainerId) of
+        ok ->
+            {reply, ok, State};
+        {error, Reason} ->
+            {reply, {error, Reason}, State}
+    end;
+
+%% 停止服务, 主动停止的时候会改变服务配置的status字段
+handle_call({kill_container, ContainerId}, _From, State = #state{}) ->
+    case docker_commands:kill_container(ContainerId) of
+        ok ->
+            {reply, ok, State};
+        {error, Reason} ->
+            {reply, {error, Reason}, State}
+    end;
+
+%% 停止服务, 主动停止的时候会改变服务配置的status字段
+handle_call(get_containers, _From, State = #state{}) ->
+    case docker_commands:get_containers() of
+        {ok, Containers} ->
+            {reply, {ok, Containers}, State};
+        {error, Reason} ->
+            {reply, {error, Reason}, State}
+    end;
+
+%% 停止服务, 主动停止的时候会改变服务配置的status字段
+handle_call({remove_container, ContainerId}, _From, State = #state{}) ->
+    case docker_commands:remove_container(ContainerId) of
+        ok ->
+            {reply, ok, State};
+        {error, Reason} ->
+            {reply, {error, Reason}, State}
+    end;
+
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info({'DOWN', _Ref, process, TaskPid, Reason}, State = #state{task_map = TaskMap}) ->
+    case maps:take(TaskPid, TaskMap) of
+        error ->
+            {noreply, State};
+        {TaskId, NTaskMap} ->
+            case Reason of
+                normal ->
+                    lager:debug("[docker_manager] task_id: ~p, exit normal", [TaskId]),
+                    ok;
+                Error0 ->
+                    Error = iolist_to_binary(io_lib:format("~p", [Error0])),
+                    efka_remote_agent:task_event_stream(TaskId, <<"error">>, <<"任务失败: "/utf8, Error/binary>>),
+                    efka_remote_agent:close_task_event_stream(TaskId, <<"task exited">>),
+                    lager:notice("[docker_manager] task_id: ~p, exit with error: ~p", [TaskId, Error]),
+                    ok
+            end,
+            {noreply, State#state{task_map = NTaskMap}}
+    end;
+
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================

apps/efka/src/efka.app.src

@@ -5,11 +5,14 @@
   {mod, {efka_app, []}},
   {applications,
    [
-    %sync,
-    docker,
+    sync,
+    jiffy,
+    parse_trans,
+    lager,
     cowboy,
     ranch,
     crypto,
+    gun,
     cowlib,
     inets,
     ssl,

apps/efka/src/efka_app.erl

@@ -9,30 +9,32 @@
 
 -export([start/2, stop/1]).
 
--spec start(term(), term()) -> {ok, pid()} | {error, term()}.
 start(_StartType, _StartArgs) ->
     io:setopts([{encoding, unicode}]),
+    ensure_upload_dir(),
     %% 加速内存的回收
     erlang:system_flag(fullsweep_after, 16),
-    start_websocket_server(),
+    start_http_server(),
 
     efka_sup:start_link().
 
--spec stop(term()) -> ok.
 stop(_State) ->
     ok.
 
 %% 微服务和efka之间通过websocket协议通讯
--spec start_websocket_server() -> ok.
-start_websocket_server() ->
-    {ok, Props} = application:get_env(efka, websocket_server),
+start_http_server() ->
+    {ok, Props} = application:get_env(efka, http_server),
     Acceptors = proplists:get_value(acceptors, Props, 50),
     MaxConnections = proplists:get_value(max_connections, Props, 10240),
     Backlog = proplists:get_value(backlog, Props, 1024),
     Port = proplists:get_value(port, Props),
 
     Dispatcher = cowboy_router:compile([
-        {'_', [{"/ws", efka_service_channel, []}]}
+        {'_', [
+            {"/ws", ws_channel, []},
+            {"/files/[...]", cowboy_static, {dir, "/usr/local/code/downloads"}},
+            {"/upload", upload_channel, []}
+        ]}
     ]),
 
     TransOpts = [
@@ -43,4 +45,13 @@ start_websocket_server() ->
     ],
     {ok, Pid} = cowboy:start_clear(ws_listener, TransOpts, #{env => #{dispatch => Dispatcher}}),
 
-    logger:debug("[efka_app] websocket server start at: ~p, pid is: ~p", [Port, Pid]).
+    lager:debug("[efka_app] websocket server start at: ~p, pid is: ~p", [Port, Pid]).
+
+ensure_upload_dir() ->
+    {ok, UploadDir} = application:get_env(efka, upload_dir),
+    case filelib:is_dir(UploadDir) of
+        true ->
+            ok;
+        false ->
+            ok = file:make_dir(UploadDir)
+    end.

apps/efka/src/efka_logger.erl

@@ -29,11 +29,10 @@
 %%% API
 %%%===================================================================
 
--spec write(Data :: binary()) -> ok.
+-spec write(Data :: binary()) -> no_return().
 write(Data) when is_binary(Data) ->
     gen_server:cast(?SERVER, {write, Data}).
 
--spec write_lines(Lines :: [binary()]) -> ok.
 write_lines(Lines) when is_list(Lines) ->
     gen_server:cast(?SERVER, {write_lines, Lines}).
 
@@ -138,13 +137,11 @@ code_change(_OldVsn, State = #state{}, _Extra) ->
 %%% Internal functions
 %%%===================================================================
 
--spec format(binary() | [iodata()]) -> binary().
 format(Data) when is_binary(Data) ->
     iolist_to_binary(Data);
 format(Items) when is_list(Items) ->
     iolist_to_binary(lists:join(<<"\t">>, Items)).
 
--spec time_prefix() -> binary().
 time_prefix() ->
     {{Y, M, D}, {H, I, S}} = calendar:local_time(),
     iolist_to_binary(io_lib:format("[~b-~2..0b-~2..0b ~2..0b:~2..0b:~2..0b]", [Y, M, D, H, I, S])).
@@ -156,7 +153,6 @@ make_file(LogFile) when is_list(LogFile) ->
     RootDir = code:root_dir() ++ "/log/",
     lists:flatten(RootDir ++ LogFile ++ "." ++ Suffix).
 
--spec ensure_dir() -> ok | {error, term()}.
 ensure_dir() ->
     RootDir = code:root_dir() ++ "/log/",
     case filelib:is_dir(RootDir) of
@@ -176,4 +172,4 @@ get_date() ->
 -spec maybe_new_file(Date :: calendar:date()) -> boolean().
 maybe_new_file({Y, M, D}) ->
     {{Y0, M0, D0}, _} = calendar:local_time(),
-    not (Y =:= Y0 andalso M =:= M0 andalso D =:= D0).
+    not (Y =:= Y0 andalso M =:= M0 andalso D =:= D0).

apps/efka/src/efka_remote_agent.erl

@@ -0,0 +1,390 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 21. 5月 2025 18:38
+%%%-------------------------------------------------------------------
+-module(efka_remote_agent).
+-author("anlicheng").
+-include("message.hrl").
+-include("efka_tables.hrl").
+
+-behaviour(gen_statem).
+
+%% API
+-export([start_link/0]).
+-export([metric_data/2, ping/13, task_event_stream/3, close_task_event_stream/2]).
+
+%% gen_statem callbacks
+-export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]).
+
+-define(SERVER, ?MODULE).
+
+%% 标记当前agent的状态，只有在 activated 状态下才可以正常的发送数据
+-define(STATE_DENIED, denied).
+-define(STATE_CONNECTING, connecting).
+-define(STATE_AUTH, auth).
+%% 不能推送消息到服务，但是可以接受服务器的部分指令
+-define(STATE_RESTRICTED, restricted).
+%% 激活状态下
+-define(STATE_ACTIVATED, activated).
+
+-record(state, {
+    transport_pid :: undefined | pid(),
+    transport_ref :: undefined | reference()
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+%% 发送数据
+-spec metric_data(RouteKey :: binary(), Metric :: binary()) -> no_return().
+metric_data(RouteKey, Metric) when is_binary(RouteKey), is_binary(Metric) ->
+    gen_statem:cast(?SERVER, {metric_data, RouteKey, Metric}).
+
+-spec task_event_stream(TaskId :: integer(), Type :: binary(), Stream :: binary()) -> no_return().
+task_event_stream(TaskId, Type, Stream) when is_integer(TaskId), is_binary(Type), is_binary(Stream) ->
+    gen_statem:cast(?SERVER, {task_event_stream, TaskId, Type, Stream}).
+
+-spec close_task_event_stream(TaskId :: integer(), Reason :: binary()) -> no_return().
+close_task_event_stream(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) ->
+    gen_statem:cast(?SERVER, {close_task_event_stream, TaskId, Reason}).
+
+ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) ->
+    gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}).
+
+%% @doc Creates a gen_statem process which calls Module:init/1 to
+%% initialize. To ensure a synchronized start-up procedure, this
+%% function does not return until Module:init/1 has returned.
+start_link() ->
+    gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_statem callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Whenever a gen_statem is started using gen_statem:start/[3,4] or
+%% gen_statem:start_link/[3,4], this function is called by the new
+%% process to initialize.
+init([]) ->
+    erlang:start_timer(0, self(), create_transport),
+    {ok, ?STATE_DENIED, #state{}}.
+
+%% @private
+%% @doc This function is called by a gen_statem when it needs to find out
+%% the callback mode of the callback module.
+callback_mode() ->
+    handle_event_function.
+
+%% @private
+%% @doc If callback_mode is handle_event_function, then whenever a
+%% gen_statem receives an event from call/2, cast/2, or as a normal
+%% process message, this function is called.
+
+%% 异步发送数据, 连接存在时候直接发送；否则缓存到mnesia
+handle_event(cast, {metric_data, RouteKey, Metric}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    Packet = message_codec:encode(?MESSAGE_DATA, #data{
+        route_key = RouteKey,
+        metric = Metric
+    }),
+    efka_transport:send(TransportPid, Packet),
+    {keep_state, State};
+
+handle_event(cast, {metric_data, RouteKey, Metric}, _, State) ->
+    Packet = message_codec:encode(?MESSAGE_DATA, #data{
+        route_key = RouteKey,
+        metric = Metric
+    }),
+    ok = cache_model:insert(Packet),
+    {keep_state, State};
+
+handle_event(cast, {task_event_stream, TaskId, Type, Stream}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    lager:debug("[efka_remote_agent] event_stream task_id: ~p, stream: ~ts", [TaskId, Stream]),
+    EventPacket = message_codec:encode(?MESSAGE_EVENT_STREAM, #task_event_stream{
+        task_id = TaskId,
+        type = Type,
+        stream = Stream
+    }),
+    efka_transport:send(TransportPid, EventPacket),
+    {keep_state, State};
+
+handle_event(cast, {close_task_event_stream, TaskId, Reason}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    EventPacket = message_codec:encode(?MESSAGE_EVENT_STREAM, #task_event_stream{
+        task_id = TaskId,
+        type = <<"close">>,
+        stream = Reason
+    }),
+    efka_transport:send(TransportPid, EventPacket),
+    {keep_state, State};
+
+%% 其他情况下直接忽略
+handle_event(cast, {task_event_stream, _TaskId, _Stream}, _, State = #state{}) ->
+    {keep_state, State};
+
+%handle_event(cast, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}, ?STATE_ACTIVATED,
+%    State = #state{transport_pid = TransportPid}) ->
+%
+%    Ping = message_pb:encode_msg(#ping{
+%        adcode = AdCode,
+%        boot_time = BootTime,
+%        province = Province,
+%        city = City,
+%        efka_version = EfkaVersion,
+%        kernel_arch = KernelArch,
+%        ips = Ips,
+%        cpu_core = CpuCore,
+%        cpu_load = CpuLoad,
+%        cpu_temperature = CpuTemperature,
+%        disk = Disk,
+%        memory = Memory,
+%        interfaces = Interfaces
+%    }),
+%    efka_transport:send(TransportPid, ?METHOD_PING, Ping),
+%    {keep_state, State};
+
+%% 异步建立到服务器的连接
+handle_event(info, {timeout, _, create_transport}, ?STATE_DENIED, State) ->
+    {ok, Props} = application:get_env(efka, tls_server),
+    Host = proplists:get_value(host, Props),
+    Port = proplists:get_value(port, Props),
+    {ok, {TransportPid, TransportRef}} = efka_transport:start_monitor(self(), Host, Port),
+    efka_transport:connect(TransportPid),
+
+    {next_state, ?STATE_CONNECTING, State#state{transport_pid = TransportPid, transport_ref = TransportRef}};
+
+handle_event(info, {connect_reply, Reply}, ?STATE_CONNECTING, State = #state{transport_pid = TransportPid}) ->
+    case Reply of
+        ok ->
+            AuthBin = auth_request(),
+            efka_transport:auth_request(TransportPid, AuthBin),
+            {next_state, ?STATE_AUTH, State};
+        {error, Reason} ->
+            lager:debug("[efka_remote_agent] connect failed, error: ~p, pid: ~p", [Reason, TransportPid]),
+            efka_transport:stop(TransportPid),
+            {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
+    end;
+
+handle_event(info, {auth_reply, Reply}, ?STATE_AUTH, State = #state{transport_pid = TransportPid}) ->
+    case Reply of
+        {ok, #auth_reply{code = Code, payload = Message}} ->
+            case Code of
+                0 ->
+                    lager:debug("[efka_remote_agent] auth success, message: ~p", [Message]),
+                    {next_state, ?STATE_ACTIVATED, State, [{next_event, info, flush_cache}]};
+                1 ->
+                    %% 主机在后台的授权未通过；此时agent不能推送数据给云端服务器，但是云端服务器可以推送命令给agent
+                    %% socket的连接状态需要维持
+                    lager:debug("[efka_remote_agent] auth denied, message: ~p", [Message]),
+                    {next_state, ?STATE_RESTRICTED, State};
+                2 ->
+                    % 其他类型的错误，需要间隔时间重试
+                    lager:debug("[efka_remote_agent] auth failed, message: ~p", [Message]),
+                    efka_transport:stop(TransportPid),
+                    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}};
+                _ ->
+                    % 其他类型的错误，需要间隔时间重试
+                    lager:debug("[efka_remote_agent] auth failed, invalid message"),
+                    efka_transport:stop(TransportPid),
+                    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
+            end;
+        {error, Reason} ->
+            lager:debug("[efka_remote_agent] auth_request failed, error: ~p", [Reason]),
+            efka_transport:stop(TransportPid),
+            {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}
+    end;
+
+%% 将缓存中的数据推送到服务器端
+handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    case cache_model:fetch_next() of
+        {ok, {Id, Packet}} ->
+            efka_transport:send(TransportPid, Packet),
+            cache_model:delete(Id),
+            {keep_state, State, [{next_event, info, flush_cache}]};
+        error ->
+            {keep_state, State}
+    end;
+handle_event(info, flush_cache, _, State) ->
+    {keep_state, State};
+
+%% 云端服务器推送了消息
+%% 激活消息
+
+%% 微服务部署
+handle_event(info, {server_rpc, PacketId, #jsonrpc_request{method = <<"get_containers">>}}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    case docker_manager:get_containers() of
+        {ok, Containers} ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_success(Containers));
+        {error, Reason} when is_binary(Reason) ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_error(-1, Reason))
+    end,
+    {keep_state, State};
+
+%% 微服务部署
+handle_event(info, {server_rpc, PacketId, #jsonrpc_request{method = <<"deploy">>, params = Params}}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #{<<"task_id">> := TaskId, <<"config">> := Config} = Params,
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    case docker_manager:deploy(TaskId, Config) of
+        ok ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_success(<<"ok">>));
+        {error, Reason} when is_binary(Reason) ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_error(-1, Reason))
+    end,
+    {keep_state, State};
+
+%% 启动微服务
+handle_event(info, {server_rpc, PacketId, #jsonrpc_request{method = <<"start_container">>, params = Params}}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #{<<"container_name">> := ContainerName} = Params,
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    case docker_manager:start_container(ContainerName) of
+        ok ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_success(<<"ok">>));
+        {error, Reason} when is_binary(Reason) ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_error(-1, Reason))
+    end,
+    {keep_state, State};
+
+%% 停止微服务
+handle_event(info, {server_rpc, PacketId, #jsonrpc_request{method = <<"stop_container">>, params = Params}}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #{<<"container_name">> := ContainerName} = Params,
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    case docker_manager:stop_container(ContainerName) of
+        ok ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_success(<<"ok">>));
+        {error, Reason} when is_binary(Reason) ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_error(-1, Reason))
+    end,
+    {keep_state, State};
+
+handle_event(info, {server_rpc, PacketId, #jsonrpc_request{method = <<"kill_container">>, params = Params}}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #{<<"container_name">> := ContainerName} = Params,
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    case docker_manager:kill_container(ContainerName) of
+        ok ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_success(<<"ok">>));
+        {error, Reason} when is_binary(Reason) ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_error(-1, Reason))
+    end,
+    {keep_state, State};
+
+handle_event(info, {server_rpc, PacketId, #jsonrpc_request{method = <<"remove_container">>, params = Params}}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #{<<"container_name">> := ContainerName} = Params,
+    %% 短暂的等待，efka_inetd收到消息后就立即返回了
+    case docker_manager:remove_container(ContainerName) of
+        ok ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_success(<<"ok">>));
+        {error, Reason} when is_binary(Reason) ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_error(-1, Reason))
+    end,
+    {keep_state, State};
+
+%% config.json配置信息
+handle_event(info, {server_rpc, PacketId, #jsonrpc_request{method = <<"config_container">>, params = Params}}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+    #{<<"container_name">> := ContainerName, <<"config">> := Config} = Params,
+    case docker_manager:config_container(ContainerName, Config) of
+        ok ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_success(<<"ok">>));
+        {error, Reason} ->
+            efka_transport:rpc_reply(TransportPid, PacketId, reply_error(-1, Reason))
+    end,
+    {keep_state, State};
+
+%% 处理task_log
+%handle_event(info, {server_async_call, PacketId, <<?PUSH_TASK_LOG:8, TaskLogBin/binary>>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) ->
+%    #fetch_task_log{task_id = TaskId} = message_pb:decode_msg(TaskLogBin, fetch_task_log),
+%    lager:debug("[efka_remote_agent] get task_log request: ~p", [TaskId]),
+%    {ok, Logs} = efka_inetd_task_log:get_logs(TaskId),
+%    Reply = case length(Logs) > 0 of
+%                true ->
+%                    Result = iolist_to_binary(jiffy:encode(Logs, [force_utf8])),
+%                    #async_call_reply{code = 1, result = Result};
+%                false ->
+%                    #async_call_reply{code = 1, result = <<"[]">>}
+%            end,
+%    efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)),
+%
+%    {keep_state, State};
+
+%% 处理命令
+handle_event(info, {server_cast, #command{command_type = ?COMMAND_AUTH, command = Auth0}}, StateName, State = #state{transport_pid = TransportPid}) ->
+    Auth = binary_to_integer(Auth0),
+    case {Auth, StateName} of
+        {1, ?STATE_ACTIVATED} ->
+            {keep_state, State};
+        {1, ?STATE_DENIED} ->
+            %% 重新激活, 需要重新校验
+            AuthRequestBin = auth_request(),
+            efka_transport:auth_request(TransportPid, AuthRequestBin),
+            {next_state, ?STATE_AUTH, State};
+        {0, _} ->
+            %% 这个时候的主机应该是受限制的状态，不允许发送消息；但是能够接受服务器推送的消息
+            {next_state, ?STATE_RESTRICTED, State}
+    end;
+
+%% 处理Pub/Sub机制
+handle_event(info, {server_cast, #pub{topic = Topic, qos = Qos, content = Content}}, ?STATE_ACTIVATED, State) ->
+    lager:debug("[efka_remote_agent] get pub topic: ~p, qos: ~p, content: ~p", [Topic, Qos, Content]),
+    %% 消息发送到订阅系统
+    efka_subscription:publish(Topic, Qos, Content),
+    {keep_state, State};
+
+%% transport进程退出
+handle_event(info, {'DOWN', MRef, process, TransportPid, Reason}, _, State = #state{transport_ref = MRef}) ->
+    lager:debug("[efka_remote_agent] transport pid: ~p, exit with reason: ~p", [TransportPid, Reason]),
+    erlang:start_timer(5000, self(), create_transport),
+    {next_state, ?STATE_DENIED, State#state{transport_pid = undefined, transport_ref = undefined}}.
+
+%% @private
+%% @doc This function is called by a gen_statem when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_statem terminates with
+%% Reason. The return value is ignored.
+terminate(_Reason, _StateName, _State = #state{transport_pid = TransportPid}) ->
+    case is_pid(TransportPid) andalso is_process_alive(TransportPid) of
+        true ->
+            efka_transport:stop(TransportPid);
+        false ->
+            ok
+    end,
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+code_change(_OldVsn, StateName, State = #state{}, _Extra) ->
+    {ok, StateName, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec auth_request() -> binary().
+auth_request() ->
+    {ok, AuthInfo} = application:get_env(efka, auth),
+    UUID = proplists:get_value(uuid, AuthInfo),
+    Username = proplists:get_value(username, AuthInfo),
+    Salt = proplists:get_value(salt, AuthInfo),
+    Token = proplists:get_value(token, AuthInfo),
+
+    message_codec:encode(?MESSAGE_AUTH_REQUEST, #auth_request{
+        uuid = unicode:characters_to_binary(UUID),
+        username = unicode:characters_to_binary(Username),
+        salt = unicode:characters_to_binary(Salt),
+        token = unicode:characters_to_binary(Token),
+        timestamp = efka_util:timestamp()
+    }).
+
+-spec reply_success(Result :: any()) -> binary().
+reply_success(Result) ->
+    message_codec:encode(?MESSAGE_JSONRPC_REPLY, #jsonrpc_reply{result = Result}).
+
+-spec reply_error(Code :: integer(), Message :: binary()) -> binary().
+reply_error(Code, Message) when is_integer(Code), is_binary(Message) ->
+    Error = #{
+        <<"code">> => Code,
+        <<"message">> => Message
+    },
+    message_codec:encode(?MESSAGE_JSONRPC_REPLY, #jsonrpc_reply{error = Error}).

apps/efka/src/efka_service.erl

@@ -40,11 +40,11 @@ get_name(ServiceId) when is_binary(ServiceId) ->
 get_pid(ServiceId) when is_binary(ServiceId) ->
     whereis(get_name(ServiceId)).
 
--spec metric_data(Pid :: pid(), RouteKey :: binary(), Metric :: binary()) -> ok.
+-spec metric_data(Pid :: pid(), RouteKey :: binary(), Metric :: binary()) -> no_return().
 metric_data(Pid, RouteKey, Metric) when is_pid(Pid), is_binary(RouteKey),  is_binary(Metric) ->
     gen_server:cast(Pid, {metric_data, RouteKey, Metric}).
 
--spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> ok.
+-spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> no_return().
 send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) ->
     gen_server:cast(Pid, {send_event, EventType, Params}).
 
@@ -69,7 +69,7 @@ start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) ->
     {stop, Reason :: term()} | ignore).
 init([ServiceId]) ->
     %% supervisor进程通过exit(ChildPid, shutdown)调用的时候，确保terminate函数被调用
-    logger:debug("[efka_service] service_id: ~p, started", [ServiceId]),
+    lager:debug("[efka_service] service_id: ~p, started", [ServiceId]),
     {ok, #state{service_id = ServiceId}}.
 
 %% @private
@@ -87,7 +87,7 @@ handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = Ol
     case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of
         false ->
             erlang:monitor(process, ChannelPid),
-            logger:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
+            lager:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]),
             {reply, ok, State#state{channel_pid = ChannelPid}};
         true ->
             {reply, {error, <<"channel exists">>}, State}
@@ -103,8 +103,8 @@ handle_call(_Request, _From, State = #state{}) ->
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
 handle_cast({metric_data, RouteKey, Metric}, State = #state{service_id = ServiceId}) ->
-    logger:debug("[efka_service] metric_data service_id: ~p, route_key: ~p, metric data: ~p", [ServiceId, RouteKey, Metric]),
-    efka_iot_client:metric_data(RouteKey, Metric),
+    lager:debug("[efka_service] metric_data service_id: ~p, route_key: ~p, metric data: ~p", [ServiceId, RouteKey, Metric]),
+    efka_remote_agent:metric_data(RouteKey, Metric),
     {noreply, State};
 
 handle_cast(_Request, State = #state{}) ->
@@ -118,7 +118,7 @@ handle_cast(_Request, State = #state{}) ->
     {stop, Reason :: term(), NewState :: #state{}}).
 %% 处理channel进程的退出
 handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) ->
-    logger:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
+    lager:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]),
     {noreply, State#state{channel_pid = undefined}}.
 
 %% @private
@@ -129,7 +129,7 @@ handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_
 -spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
     State :: #state{}) -> term()).
 terminate(Reason, _State = #state{service_id = ServiceId}) ->
-    logger:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
+    lager:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]),
     ok.
 
 %% @private
@@ -142,4 +142,4 @@ code_change(_OldVsn, State = #state{}, _Extra) ->
 
 %%%===================================================================
 %%% Internal functions
-%%%===================================================================
+%%%===================================================================

apps/efka/src/efka_service_sup.erl

@@ -39,7 +39,6 @@ start_link() ->
 %% this function is called by the new process to find out about
 %% restart strategy, maximum restart frequency and child
 %% specifications.
--spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
 init([]) ->
     SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
     {ok, {SupFlags, []}}.
@@ -65,7 +64,6 @@ stop_service(ServiceId) when is_binary(ServiceId) ->
     supervisor:terminate_child(?MODULE, ChildId),
     supervisor:delete_child(?MODULE, ChildId).
 
--spec child_spec(binary()) -> supervisor:child_spec().
 child_spec(ServiceId) when is_binary(ServiceId) ->
     Name = efka_service:get_name(ServiceId),
     #{
@@ -75,4 +73,4 @@ child_spec(ServiceId) when is_binary(ServiceId) ->
         shutdown => 5000,
         type => worker,
         modules => ['efka_service']
-    }.
+    }.

apps/efka/src/efka_stream.erl

@@ -0,0 +1,164 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 13. 11月 2025 10:57
+%%%-------------------------------------------------------------------
+-module(efka_stream).
+-author("anlicheng").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_monitor/1]).
+-export([setup/3, data/2, finish/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+
+-record(state, {
+    parent_pid :: pid(),
+    ref :: reference(),
+    file_size = 0 :: integer(),
+    acc_size = 0 :: integer(),
+    real_file :: undefined | string(),
+    io_device :: undefined | file:fd()
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec setup(StreamPid :: pid(), FileName :: string(), FileSize :: integer()) -> {ok, Path :: string()}.
+setup(StreamPid, FileName, FileSize) when is_pid(StreamPid), is_list(FileName), is_integer(FileSize) ->
+    gen_server:call(StreamPid, {setup, FileName, FileSize}).
+
+-spec data(StreamPid :: pid(), ChunkData :: binary()) -> no_return().
+data(StreamPid, ChunkData) when is_pid(StreamPid), is_binary(ChunkData) ->
+    gen_server:cast(StreamPid, {data, ChunkData}).
+
+-spec finish(StreamPid :: pid()) -> no_return().
+finish(StreamPid) when is_pid(StreamPid) ->
+    gen_server:cast(StreamPid, finish).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_monitor(ParentPid :: pid()) ->
+    {ok, {Pid :: pid(), MonRef :: reference()}} | ignore | {error, Reason :: term()}).
+start_monitor(ParentPid) when is_pid(ParentPid) ->
+    gen_server:start_monitor(?MODULE, [ParentPid], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([ParentPid]) ->
+    Ref = erlang:monitor(process, ParentPid),
+    {ok, #state{parent_pid = ParentPid, ref = Ref}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call({setup, FileName, FileSize}, _From, State = #state{}) ->
+    {RealFileName, Path} = make_file(filename:basename(FileName)),
+    {ok, IoDevice} = file:open(RealFileName, [write]),
+
+    {reply, {ok, Path}, State#state{io_device = IoDevice, real_file = RealFileName, file_size = FileSize, acc_size = 0}}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast({data, ChunkData}, State = #state{io_device = IoDevice, acc_size = AccSize}) ->
+    Data = base64:decode(ChunkData),
+    Len = byte_size(Data),
+
+    ok = file:write(IoDevice, Data),
+    {noreply, State#state{acc_size = AccSize + Len}};
+handle_cast(finish, State = #state{parent_pid = ParentPid, io_device = IoDevice, acc_size = AccSize, file_size = FileSize, real_file = RealFile}) ->
+    case AccSize == FileSize of
+        true ->
+            ok = file:close(IoDevice),
+            ParentPid ! {stream_reply, self(), ok};
+        false ->
+            ok = file:close(IoDevice),
+            ok = file:delete(RealFile),
+            ParentPid ! {stream_reply, self(), invalid}
+    end,
+    {stop, normal, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info({'DOWN', Ref, process, Pid, normal}, State = #state{ref = Ref, parent_pid = Pid}) ->
+    {noreply, State};
+handle_info({'DOWN', Ref, process, Pid, Reason}, State = #state{ref = Ref, parent_pid = Pid, io_device = IoDevice, real_file = RealFile}) ->
+    lager:debug("[efka_stream] ws_channel close with reason: ~p", [Reason]),
+    case IoDevice =:= undefined of
+        true ->
+            ok;
+        false ->
+            ok = file:close(IoDevice),
+            RealFile /= undefined andalso file:delete(RealFile)
+    end,
+    {stop, normal, State};
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec make_file(Basename :: string()) -> {string(), string()}.
+make_file(Basename) when is_list(Basename) ->
+    {ok, UploadDir} = application:get_env(efka, upload_dir),
+    {{Y, M, D}, _} = calendar:local_time(),
+    DateDir = io_lib:format("~p-~p-~p", [Y, M, D]),
+    BaseDir = UploadDir ++ DateDir,
+    case filelib:is_dir(BaseDir) of
+        true ->
+            ok;
+        false ->
+            ok = file:make_dir(BaseDir)
+    end,
+    Path = DateDir ++ "/" ++ Basename,
+
+    {UploadDir ++ Path, Path}.

apps/efka/src/efka_subscription.erl

@@ -13,7 +13,7 @@
 
 %% API
 -export([start_link/0]).
--export([subscribe/2, unsubscribe_all/1, publish/3, debug_info/0]).
+-export([subscribe/2, publish/3, debug_info/0]).
 -export([match_components/2, is_valid_components/1, of_components/1]).
 
 %% gen_server callbacks
@@ -47,14 +47,10 @@
 subscribe(Topic, SubscriberPid) when is_binary(Topic), is_pid(SubscriberPid) ->
     gen_server:call(?SERVER, {subscribe, Topic, SubscriberPid}).
 
--spec publish(Topic :: binary(), Qos :: integer(), Content :: binary()) -> ok.
+-spec publish(Topic :: binary(), Qos :: integer(), Content :: binary()) -> no_return().
 publish(Topic, Qos, Content) when is_binary(Topic), is_integer(Qos), is_binary(Content) ->
     gen_server:cast(?SERVER, {publish, Topic, Qos, Content}).
 
--spec unsubscribe_all(pid()) -> ok.
-unsubscribe_all(SubscriberPid) when is_pid(SubscriberPid) ->
-    gen_server:call(?SERVER, {unsubscribe_all, SubscriberPid}).
-
 -spec debug_info() -> {ok, Info :: map()}.
 debug_info() ->
     gen_server:call(?SERVER, debug_info).
@@ -92,27 +88,15 @@ handle_call({subscribe, Topic, SubscriberPid}, _From, State = #state{subscribers
     Components = of_components(Topic),
     case is_valid_components(Components) of
         true ->
-            case has_subscription(Topic, SubscriberPid, Subscribers) of
-                true ->
-                    {reply, ok, State};
-                false ->
-                    Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)},
-                    %% 只有首次看到该pid时才建立monitor，避免重复monitor
-                    case has_subscriber_pid(SubscriberPid, Subscribers) of
-                        true ->
-                            ok;
-                        false ->
-                            erlang:monitor(process, SubscriberPid)
-                    end,
-                    %% 处理遗留的消息
-                    RestRemainingMessages = dispatch_remaining_messages(Sub, RemainingMessages),
-                    {reply, ok, State#state{subscribers = Subscribers ++ [Sub], remaining_messages = RestRemainingMessages}}
-            end;
+            Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)},
+            %% 建立到SubscriberPid的monitor，进程退出需要清理订阅
+            erlang:monitor(process, SubscriberPid),
+            %% 处理遗留的消息
+            RestRemainingMessages = dispatch_remaining_messages(Sub, RemainingMessages),
+            {reply, ok, State#state{subscribers = Subscribers ++ [Sub], remaining_messages = RestRemainingMessages}};
         false ->
             {reply, {error, <<"invalid topic name">>}, State}
     end;
-handle_call({unsubscribe_all, SubscriberPid}, _From, State = #state{subscribers = Subscribers}) ->
-    {reply, ok, State#state{subscribers = remove_subscriber_pid(SubscriberPid, Subscribers)}};
 handle_call(debug_info, _From, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
     Info = #{
         subscribes => Subscribers,
@@ -129,14 +113,14 @@ handle_call(debug_info, _From, State = #state{subscribers = Subscribers, remaini
 %% 发布消息
 handle_cast({publish, Topic, Qos, Content}, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) ->
     MatchedSubscribers = match_subscribers(Subscribers, Topic),
-    logger:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]),
-    case MatchedSubscribers of
-        [_|_] ->
+    lager:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]),
+    case length(MatchedSubscribers) > 0 of
+        true ->
             broadcast(Topic, Content, MatchedSubscribers),
             {noreply, State};
-        [] when Qos =:= 0 ->
+        false when Qos =:= 0 ->
             {noreply, State};
-        [] ->
+        false ->
             {noreply, State#state{remaining_messages = [{Topic, Content}|RemainingMessages]}}
     end.
 
@@ -147,12 +131,12 @@ handle_cast({publish, Topic, Qos, Content}, State = #state{subscribers = Subscri
     {noreply, NewState :: #state{}, timeout() | hibernate} |
     {stop, Reason :: term(), NewState :: #state{}}).
 handle_info({'DOWN', _Ref, process, SubscriberPid, Reason}, State = #state{subscribers = Subscribers}) ->
-    logger:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]),
-    NSubscribers = remove_subscriber_pid(SubscriberPid, Subscribers),
+    lager:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]),
+    NSubscribers = lists:filter(fun(#subscriber{subscriber_pid = Pid0}) -> SubscriberPid /= Pid0 end, Subscribers),
     {noreply, State#state{subscribers = NSubscribers}};
 
 handle_info(Info, State = #state{}) ->
-    logger:debug("[efka_subscription] get unknown info: ~p", [Info]),
+    lager:debug("[efka_subscription] get unknown info: ~p", [Info]),
     {noreply, State}.
 
 %% @private
@@ -194,17 +178,6 @@ match_subscribers(Subscribers, Topic) when is_list(Subscribers), is_binary(Topic
 contain_channel(Pid, Subscribers) when is_pid(Pid), is_list(Subscribers) ->
     lists:search(fun(#subscriber{subscriber_pid = Pid0}) -> Pid == Pid0 end, Subscribers) /= false.
 
--spec has_subscriber_pid(pid(), [#subscriber{}]) -> boolean().
-has_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) ->
-    lists:any(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =:= SubscriberPid0 end, Subscribers).
-
--spec has_subscription(binary(), pid(), [#subscriber{}]) -> boolean().
-has_subscription(Topic, SubscriberPid, Subscribers)
-  when is_binary(Topic), is_pid(SubscriberPid), is_list(Subscribers) ->
-    lists:any(fun(#subscriber{topic = Topic0, subscriber_pid = SubscriberPid0}) ->
-        Topic =:= Topic0 andalso SubscriberPid =:= SubscriberPid0
-    end, Subscribers).
-
 %% 开始对比订阅的topic和发布的topic的Components信息
 %% *表示单级匹配，+表示多级匹配；+只能出现一次，并且只能在末尾
 -spec match_components(list(), list()) -> boolean().
@@ -225,7 +198,6 @@ match_components(_, _, _) ->
 of_components(Topic) when is_binary(Topic) ->
     binary:split(Topic, <<$/>>, [global]).
 
--spec is_valid_components([binary()]) -> boolean().
 is_valid_components([]) ->
     true;
 is_valid_components([<<$+>>|T]) ->
@@ -245,26 +217,21 @@ order_num([<<$+>>|_]) ->
 order_num([_|Tail]) ->
     order_num(Tail).
 
--spec broadcast(binary(), binary(), [#subscriber{}]) -> ok.
 broadcast(Topic, Content, MatchedSubscribers) ->
     lists:foreach(fun(#subscriber{subscriber_pid = SubscriberPid}) ->
         SubscriberPid ! {topic_broadcast, Topic, Content}
                   end, MatchedSubscribers).
 
--spec remove_subscriber_pid(pid(), [#subscriber{}]) -> [#subscriber{}].
-remove_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) ->
-    lists:filter(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =/= SubscriberPid0 end, Subscribers).
-
 -spec dispatch_remaining_messages(Subscriber :: #subscriber{}, RemainingMessages :: list()) -> RestRemainingMessages :: list().
 dispatch_remaining_messages(#subscriber{subscriber_pid = SubscriberPid, components = Components}, RemainingMessages) when is_list(RemainingMessages) ->
     %% 处理遗留的消息
     lists:foldl(fun({Topic0, Content0}, Acc) ->
         Components0 = of_components(Topic0),
-        case match_components(Components, Components0) of
+        case match_components(Components0, Components) of
             true ->
                 SubscriberPid ! {topic_broadcast, Topic0, Content0},
                 Acc;
             false ->
                 [{Topic0, Content0}|Acc]
         end
-                end, [], RemainingMessages).
+                end, [], RemainingMessages).

apps/efka/src/efka_sup.erl

@@ -13,7 +13,6 @@
 
 -define(SERVER, ?MODULE).
 
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
 start_link() ->
     supervisor:start_link({local, ?SERVER}, ?MODULE, []).
 
@@ -26,7 +25,6 @@ start_link() ->
 %%                  shutdown => shutdown(), % optional
 %%                  type => worker(),       % optional
 %%                  modules => modules()}   % optional
--spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}.
 init([]) ->
     SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600},
     ChildSpecs = [
@@ -48,13 +46,31 @@ init([]) ->
             modules => ['efka_service_sup']
         },
 
+        %#{
+        %    id => 'docker_events',
+        %    start => {'docker_events', start_link, []},
+        %    restart => permanent,
+        %    shutdown => 2000,
+        %    type => worker,
+        %    modules => ['docker_events']
+        %},
+
         #{
-            id => efka_service_model,
-            start => {efka_service_model, start_link, []},
+            id => cache_model,
+            start => {cache_model, start_link, []},
             restart => permanent,
             shutdown => 5000,
             type => worker,
-            modules => ['efka_service_model']
+            modules => ['cache_model']
+        },
+
+        #{
+            id => service_model,
+            start => {service_model, start_link, []},
+            restart => permanent,
+            shutdown => 5000,
+            type => worker,
+            modules => ['service_model']
         },
 
         #{
@@ -67,21 +83,21 @@ init([]) ->
         },
 
         #{
-            id => 'efka_iot_client',
-            start => {'efka_iot_client', start_link, []},
+            id => 'docker_manager',
+            start => {'docker_manager', start_link, []},
             restart => permanent,
             shutdown => 2000,
             type => worker,
-            modules => ['efka_iot_client']
+            modules => ['docker_manager']
         },
 
         #{
-            id => 'efka_iot_heartbeat',
-            start => {'efka_iot_heartbeat', start_link, []},
+            id => 'efka_remote_agent',
+            start => {'efka_remote_agent', start_link, []},
             restart => permanent,
             shutdown => 2000,
             type => worker,
-            modules => ['efka_iot_heartbeat']
+            modules => ['efka_remote_agent']
         }
 
     ],
@@ -89,3 +105,4 @@ init([]) ->
     {ok, {SupFlags, ChildSpecs}}.
 
 %% internal functions
+

apps/efka/src/efka_transport.erl

@@ -0,0 +1,198 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 20. 4月 2025 18:47
+%%%-------------------------------------------------------------------
+-module(efka_transport).
+-author("anlicheng").
+-include("message.hrl").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_monitor/3]).
+-export([connect/1, auth_request/2, send/2, rpc_reply/3, stop/1]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE). 
+
+-record(state, {
+    parent_pid :: pid(),
+    host :: string(),
+    port :: integer(),
+    socket :: undefined | ssl:sslsocket()
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec auth_request(Pid :: pid(), AuthBin :: binary()) -> no_return().
+auth_request(Pid, AuthBin) when is_pid(Pid), is_binary(AuthBin) ->
+    gen_server:cast(Pid, {auth_request, AuthBin}).
+
+-spec connect(Pid :: pid()) -> no_return().
+connect(Pid) when is_pid(Pid) ->
+    gen_server:cast(Pid, connect).
+
+-spec send(Pid :: pid(), Packet :: binary()) -> no_return().
+send(Pid, Packet) when is_pid(Pid), is_binary(Packet) ->
+    gen_server:cast(Pid, {send, Packet}).
+
+-spec rpc_reply(Pid :: pid() | undefined, PacketId :: integer(), Response :: binary()) -> no_return().
+rpc_reply(undefined, PacketId, Response) when is_integer(PacketId), is_binary(Response) ->
+    ok;
+rpc_reply(Pid, PacketId, Reply) when is_pid(Pid), is_integer(PacketId), is_binary(Reply) ->
+    gen_server:cast(Pid, {rpc_reply, PacketId, Reply}).
+
+%% 关闭的时候不一定能成功，可能关闭的时候；transport进程已经退出了
+-spec stop(Pid :: pid() | undefined) -> ok.
+stop(undefined) ->
+    ok;
+stop(Pid) when is_pid(Pid) ->
+    catch gen_server:stop(Pid, normal, 2000).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_monitor(ParentPid :: pid(), Host :: string(), Port :: integer()) ->
+    {ok, {Pid :: pid(), MRef :: reference()}} | ignore | {error, Reason :: term()}).
+start_monitor(ParentPid, Host, Port) when is_pid(ParentPid), is_list(Host), is_integer(Port) ->
+    gen_server:start_monitor(?MODULE, [ParentPid, Host, Port], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([ParentPid, Host, Port]) ->
+    {ok, #state{parent_pid = ParentPid, host = Host, port = Port, socket = undefined}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call(_Req, _From, State = #state{}) ->
+    {reply, ok, State#state{}}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 建立到目标服务器的连接
+handle_cast(connect, State = #state{host = Host, port = Port, parent_pid = ParentPid}) ->
+    SslOptions = [
+        binary,
+        {packet, 4},
+        {verify, verify_none}
+    ],
+    case ssl:connect(Host, Port, SslOptions, 5000) of
+        {ok, Socket} ->
+            ok = ssl:controlling_process(Socket, self()),
+            ParentPid ! {connect_reply, ok},
+            ping_ticker(),
+            {noreply, State#state{socket = Socket}};
+        {error, Reason} ->
+            ParentPid ! {connect_reply, {error, Reason}},
+            {noreply, State#state{socket = undefined}}
+    end;
+
+%% auth校验
+handle_cast({auth_request, AuthRequestBin}, State = #state{parent_pid = ParentPid, socket = Socket}) ->
+    PacketId = 1,
+    ok = ssl:send(Socket, <<?PACKET_REQUEST, PacketId:32, AuthRequestBin/binary>>),
+    %% 需要等待auth返回的结果
+    receive
+        {ssl, Socket, <<?PACKET_RESPONSE, PacketId:32, ReplyBin/binary>>} ->
+            {ok, #auth_reply{} = Reply} = message_codec:decode(ReplyBin),
+            ParentPid ! {auth_reply, {ok, Reply}},
+            {noreply, State};
+        {ssl, Socket, Info} ->
+            lager:warning("[efka_transport] get invalid auth_reply: ~p", [Info]),
+            ParentPid ! {auth_reply, {error, invalid_auth_reply}},
+            {noreply, State}
+    after 5000 ->
+        ParentPid ! {auth_reply, {error, timeout}},
+        {noreply, State}
+    end;
+
+handle_cast({send, Packet}, State = #state{socket = Socket}) ->
+    ok = ssl:send(Socket, <<?PACKET_CAST, Packet/binary>>),
+    {noreply, State};
+
+%% 服务push的消息的回复
+handle_cast({rpc_reply, PacketId, Reply}, State = #state{socket = Socket}) ->
+    ok = ssl:send(Socket, <<?PACKET_RESPONSE, PacketId:32, Reply/binary>>),
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+%% 服务器主动推送的数据，有packetId的是要求返回的；为0的表示不需要返回值
+handle_info({ssl, Socket, <<?PACKET_CAST, CastBin/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
+    {ok, CastRequest} = message_codec:decode(CastBin),
+    ParentPid ! {server_cast, CastRequest},
+    {noreply, State};
+
+handle_info({ssl, Socket, <<?PACKET_REQUEST, PacketId:32, RPCRequestBin/binary>>}, State = #state{socket = Socket, parent_pid = ParentPid}) ->
+    {ok, RPCRequest} = message_codec:decode(RPCRequestBin),
+    ParentPid ! {server_rpc, PacketId, RPCRequest},
+    {noreply, State};
+
+handle_info({ssl_error, Socket, Reason}, State = #state{socket = Socket}) ->
+    lager:debug("[efka_transport] ssl error: ~p", [Reason]),
+    {stop, normal, State};
+
+handle_info({ssl_closed, Socket}, State = #state{socket = Socket}) ->
+    {stop, normal, State};
+
+handle_info({timeout, _, ping_ticker}, State) ->
+    ping_ticker(),
+    {noreply, State};
+
+handle_info(Info, State = #state{}) ->
+    lager:notice("[efka_transport] get unknown info: ~p", [Info]),
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+        State :: #state{}) -> term()).
+terminate(Reason, #state{}) ->
+    lager:notice("[efka_transport] terminate with reason: ~p", [Reason]),
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+        Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+ping_ticker() ->
+    erlang:start_timer(5000, self(), ping_ticker).

apps/efka/src/efka_util.erl

@@ -16,30 +16,25 @@
 -export([json_data/1, json_error/2]).
 -export([starts_with/2, file_md5/1]).
 
--spec get_file_md5(string()) -> string().
 get_file_md5(FilePath) when is_list(FilePath) ->
     {ok, FileData} = file:read_file(FilePath),
     Md5Binary = crypto:hash(md5, FileData),
     string:lowercase(lists:flatten([io_lib:format("~2.16.0b", [X]) || X <- binary_to_list(Md5Binary)])).
 
 %% 时间，精确到毫秒
--spec timestamp_ms() -> integer().
 timestamp_ms() ->
     {Mega, Seconds, Micro} = os:timestamp(),
     (Mega * 1000000 + Seconds) * 1000 + Micro div 1000.
 
--spec timestamp() -> integer().
 timestamp() ->
     {Mega, Seconds, _Micro} = os:timestamp(),
     Mega * 1000000 + Seconds.
 
--spec number_format(integer() | float(), integer()) -> integer() | float().
 number_format(Num, _Decimals) when is_integer(Num) ->
     Num;
 number_format(Float, Decimals) when is_float(Float) ->
     list_to_float(float_to_list(Float, [{decimals, Decimals}, compact])).
 
--spec int_format(integer(), pos_integer()) -> integer().
 int_format(Num, Len) when is_integer(Num), Len > 0 ->
     S = integer_to_list(Num),
     case length(S) > Len of
@@ -55,8 +50,6 @@ chunks(List, Size) when is_list(List), is_integer(Size), Size > 0, length(List)
     [List];
 chunks(List, Size) when is_list(List), is_integer(Size), Size > 0 ->
     chunks0(List, Size, Size, [], []).
-
--spec chunks0(list(), integer(), integer(), list(), [list()]) -> [list()].
 chunks0([], _, _, [], AccTarget) ->
     lists:reverse(AccTarget);
 chunks0([], _, _, Target, AccTarget) ->
@@ -66,22 +59,19 @@ chunks0(List, Size, 0, Target, AccTarget) ->
 chunks0([Hd | Tail], Size, Num, Target, AccTarget) ->
     chunks0(Tail, Size, Num - 1, [Hd | Target], AccTarget).
 
--spec json_data(term()) -> binary().
 json_data(Data) ->
-    iolist_to_binary(json:encode(#{
+    jiffy:encode(#{
         <<"result">> => Data
-    })).
+    }, [force_utf8]).
 
--spec json_error(integer(), binary()) -> binary().
 json_error(ErrCode, ErrMessage) when is_integer(ErrCode), is_binary(ErrMessage) ->
-    iolist_to_binary(json:encode(#{
+    jiffy:encode(#{
         <<"error">> => #{
             <<"code">> => ErrCode,
             <<"message">> => ErrMessage
         }
-    })).
+    }, [force_utf8]).
 
--spec uuid() -> string().
 uuid() ->
     rand_bytes(16).
 
@@ -96,7 +86,6 @@ rand_bytes(Size) when is_integer(Size), Size > 0 ->
 md5(Str) when is_binary(Str) ->
     list_to_binary(lists:flatten([hex(X) || <<X:4>> <= erlang:md5(Str)])).
 
--spec hex(0..15) -> byte().
 hex(N) when N < 10 ->
     $0 + N;
 hex(N) ->
@@ -131,7 +120,6 @@ file_md5(FilePath) when is_list(FilePath) ->
     file:close(F),
     lists:flatten(io_lib:format("~32.16.0b", [binary:decode_unsigned(Digest)])).
 
--spec md5_loop(file:io_device(), crypto:hash_state()) -> binary().
 md5_loop(F, Context) ->
     %% 每次读取 1MB，可调整块大小
     case file:read(F, 1024 * 1024) of
@@ -140,3 +128,4 @@ md5_loop(F, Context) ->
         {ok, Bin} ->
             md5_loop(F, crypto:hash_update(Context, Bin))
     end.
+

apps/efka/src/iot/efka_iot_client.erl

@@ -1,675 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author anlicheng
-%%% @copyright (C) 2025, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 20. 4月 2026 00:00
-%%%-------------------------------------------------------------------
--module(efka_iot_client).
--author("anlicheng").
--include("efka_tables.hrl").
--include("message.hrl").
--include("message_pb.hrl").
-
--behaviour(gen_statem).
-
-%% API
--export([start_link/0]).
--export([metric_data/2, ping/13]).
--export([send_stream/2, stream_done/1]).
--export([is_activated/0, dropped_message_count/0]).
-
-%% gen_statem callbacks
--export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]).
-
--define(SERVER, ?MODULE).
-
-%% 标记当前agent的状态，只有在 activated 状态下才可以正常的发送数据
--define(STATE_DISCONNECTED, disconnected).
-%% 等待校验中
--define(STATE_AUTH, auth).
-%% 激活状态下
--define(STATE_ACTIVATED, activated).
--define(SSL_PING_INTERVAL, 30000).
--define(OUTBOX_SEGMENT_RECORD_LIMIT, 2000).
--define(OUTBOX_MAX_SEGMENTS, 5).
--define(OUTBOX_MAX_RECORD_BYTES, 16 * 1024 * 1024).
-
--record(state, {
-    socket :: undefined | ssl:sslsocket(),
-    outbox :: efka_iot_outbox:outbox(),
-    streams = #{},
-    %% 保存当前 auth 请求的 packet id，用来建立 auth 请求和响应的对应关系
-    auth_pkt_id = undefined :: undefined | pos_integer(),
-    next_pkt_id = 1 :: pos_integer(),
-    ping_timer_ref = undefined :: undefined | reference(),
-    dropped_message_count = 0 :: non_neg_integer()
-}).
-
--record(stream_state, {
-    worker_pid :: pid(),
-    monitor_ref :: reference()
-}).
-
--type stream_id() :: pos_integer().
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
-%% 发送数据
--spec metric_data(RouteKey :: binary(), Metric :: binary()) -> ok.
-metric_data(RouteKey, Metric) when is_binary(RouteKey), is_binary(Metric) ->
-    gen_statem:cast(?SERVER, {metric_data, RouteKey, Metric}).
-
--spec send_stream(StreamId :: stream_id(), Body :: term()) -> ok.
-send_stream(StreamId, Body) when is_integer(StreamId), StreamId > 0 ->
-    gen_statem:cast(?SERVER, {send_stream, StreamId, Body}).
-
--spec stream_done(StreamId :: stream_id()) -> ok.
-stream_done(StreamId) when is_integer(StreamId), StreamId > 0 ->
-    gen_statem:cast(?SERVER, {stream_done, StreamId}).
-
--spec is_activated() -> boolean().
-is_activated() ->
-    gen_statem:call(?SERVER, is_activated).
-
--spec dropped_message_count() -> non_neg_integer().
-dropped_message_count() ->
-    gen_statem:call(?SERVER, dropped_message_count).
-
--spec ping(term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term()) -> ok.
-ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) ->
-    gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}).
-
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
-start_link() ->
-    gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []).
-
-%%%===================================================================
-%%% gen_statem callbacks
-%%%===================================================================
-
--spec init(list()) -> {ok, atom(), #state{}}.
-init([]) ->
-    case efka_iot_outbox:open(outbox_options()) of
-        {ok, Outbox} ->
-            erlang:start_timer(0, self(), create_transport),
-            {ok, ?STATE_DISCONNECTED, #state{socket = undefined, outbox = Outbox}};
-        {error, Reason} ->
-            {stop, Reason}
-    end.
-
--spec callback_mode() -> handle_event_function.
-callback_mode() ->
-    handle_event_function.
-
--spec outbox_options() -> map().
-outbox_options() ->
-    {ok, DetsDir} = application:get_env(efka, dets_dir),
-    #{
-        dir => filename:join(DetsDir, "iot_outbox"),
-        segment_record_limit => ?OUTBOX_SEGMENT_RECORD_LIMIT,
-        max_segments => ?OUTBOX_MAX_SEGMENTS,
-        max_record_bytes => ?OUTBOX_MAX_RECORD_BYTES
-    }.
-
-%% 异步发送数据，连接存在时直接发送；否则写入持久化 outbox。
--spec handle_event(term(), term(), atom(), #state{}) -> term().
-handle_event(cast, {metric_data, RouteKey, Metric}, StateName, State = #state{socket = Socket}) ->
-    Packet = encode_message_frame({metric_data, RouteKey, Metric}),
-    case StateName of
-        ?STATE_ACTIVATED ->
-            ok = ssl:send(Socket, Packet),
-            {keep_state, State};
-        _ ->
-            case efka_iot_outbox:append(Packet, State#state.outbox) of
-                {ok, Outbox} ->
-                    {keep_state, State#state{outbox = Outbox}};
-                {dropped, capacity_reached, Outbox} ->
-                    logger:warning("[efka_iot_client] outbox capacity reached, drop offline metric"),
-                    {keep_state, State#state{
-                        outbox = Outbox,
-                        dropped_message_count = State#state.dropped_message_count + 1
-                    }};
-                {error, Reason} ->
-                    logger:warning("[efka_iot_client] append outbox failed, reason: ~p", [Reason]),
-                    {keep_state, State#state{dropped_message_count = State#state.dropped_message_count + 1}}
-            end
-    end;
-
-handle_event(cast, {send_stream, StreamId, Body}, ?STATE_ACTIVATED, State = #state{socket = Socket})
-    when is_integer(StreamId), StreamId > 0 ->
-    Packet = encode_stream_frame(StreamId, Body),
-    ok = ssl:send(Socket, Packet),
-    {keep_state, State};
-handle_event(cast, {send_stream, _StreamId, _Body}, _StateName, State) ->
-    {keep_state, State};
-
-handle_event(cast, {stream_done, StreamId}, _StateName, State = #state{streams = Streams})
-    when is_integer(StreamId), StreamId > 0 ->
-    case maps:take(StreamId, Streams) of
-        error ->
-            {keep_state, State};
-        {StreamState, NStreams} ->
-            demonitor_stream(StreamState),
-            {keep_state, State#state{streams = NStreams}}
-    end;
-
-%% 其他情况下直接忽略
-handle_event(cast, _, _, State = #state{}) ->
-    {keep_state, State};
-
-handle_event({call, From}, is_activated, ?STATE_ACTIVATED, State = #state{}) ->
-    {keep_state, State, [{reply, From, true}]};
-handle_event({call, From}, is_activated, _StateName, State = #state{}) ->
-    {keep_state, State, [{reply, From, false}]};
-handle_event({call, From}, dropped_message_count, _StateName, State = #state{dropped_message_count = DroppedCount}) ->
-    {keep_state, State, [{reply, From, DroppedCount}]};
-
-%% 异步建立到服务器的连接
-handle_event(info, {timeout, _, create_transport}, ?STATE_DISCONNECTED, State = #state{next_pkt_id = PktId}) ->
-    case connect_socket() of
-        {ok, Socket} ->
-            AuthPacket = auth_packet(PktId),
-            ok = ssl:send(Socket, AuthPacket),
-            logger:debug("[efka_iot_client] send auth request, packet_id: ~p", [PktId]),
-            {next_state, ?STATE_AUTH, State#state{socket = Socket, auth_pkt_id = PktId, next_pkt_id = PktId + 1}, [{state_timeout, 5000, auth_timeout}]};
-        {error, _Reason} ->
-            schedule_reconnect(),
-            {keep_state, State#state{socket = undefined}}
-    end;
-
-handle_event(state_timeout, auth_timeout, ?STATE_AUTH, State = #state{socket = Socket}) ->
-    logger:debug("[efka_iot_client] auth request timeout"),
-    disconnect(Socket),
-    schedule_reconnect(),
-    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_pkt_id = undefined}};
-
-handle_event(info, {timeout, TimerRef, ssl_ping}, ?STATE_ACTIVATED, State = #state{socket = Socket, ping_timer_ref = TimerRef}) ->
-    Packet = encode_message_frame(ping),
-    case ssl:send(Socket, Packet) of
-        ok ->
-            {keep_state, schedule_ssl_ping(State)};
-        {error, Reason} ->
-            logger:warning("[efka_iot_client] send ssl ping failed, reason: ~p", [Reason]),
-            disconnect(Socket),
-            schedule_reconnect(),
-            NState = close_all_streams({send_ping_failed, Reason}, State),
-            {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}}
-    end;
-handle_event(info, {timeout, _TimerRef, ssl_ping}, _StateName, State) ->
-    {keep_state, State};
-
-%% 将缓存中的数据推送到服务器端
-handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
-    case efka_iot_outbox:next(State#state.outbox) of
-        {ok, Seq, Packet} ->
-            ok = ssl:send(Socket, Packet),
-            case efka_iot_outbox:ack(Seq, State#state.outbox) of
-                {ok, Outbox} ->
-                    {keep_state, State#state{outbox = Outbox}, [{next_event, info, flush_cache}]};
-                {error, Reason} ->
-                    logger:warning("[efka_iot_client] ack outbox failed, seq: ~p, reason: ~p", [Seq, Reason]),
-                    {keep_state, State}
-            end;
-        eof ->
-            {keep_state, State};
-        {error, Reason} ->
-            logger:warning("[efka_iot_client] read outbox failed, reason: ~p", [Reason]),
-            {keep_state, State}
-    end;
-handle_event(info, flush_cache, _, State) ->
-    {keep_state, State};
-
-%% 处理收到的ssl消息
-handle_event(info, {ssl, Socket, PacketBin}, _, State = #state{socket = Socket}) when is_binary(PacketBin) ->
-    case decode_frame(PacketBin) of
-        {ok, Packet} ->
-            {keep_state, State, [{next_event, internal, Packet}]};
-        {error, Reason} ->
-            logger:warning("[efka_iot_client] decode packet failed: ~p, packet_size: ~p", [Reason, byte_size(PacketBin)]),
-            disconnect(Socket),
-            cancel_ssl_ping(State),
-            schedule_reconnect(),
-            NState = close_all_streams(bad_packet, State),
-            {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}}
-    end;
-handle_event(info, {ssl_error, Socket, Reason}, _, State = #state{}) ->
-    logger:debug("[efka_iot_client] ssl error: ~p", [Reason]),
-    disconnect(Socket),
-    cancel_ssl_ping(State),
-    schedule_reconnect(),
-    NState = close_all_streams(ssl_error, State),
-    {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}};
-handle_event(info, {ssl_closed, Socket}, _, State = #state{}) ->
-    logger:debug("[efka_iot_client] ssl closed"),
-    disconnect(Socket),
-    cancel_ssl_ping(State),
-    schedule_reconnect(),
-    NState = close_all_streams(ssl_closed, State),
-    {next_state, ?STATE_DISCONNECTED, NState#state{socket = undefined, auth_pkt_id = undefined, ping_timer_ref = undefined}};
-
-handle_event(info, {'DOWN', MonitorRef, process, WorkerPid, Reason}, _StateName, State = #state{streams = Streams}) ->
-    case take_stream_by_monitor(MonitorRef, WorkerPid, Streams) of
-        error ->
-            {keep_state, State};
-        {StreamId, StreamState, NStreams} ->
-            demonitor_stream(StreamState),
-            case Reason of
-                normal ->
-                    {keep_state, State#state{streams = NStreams}};
-                _ ->
-                    logger:warning("[efka_iot_client] stream worker down, stream_id: ~p, reason: ~p", [StreamId, Reason]),
-                    case State#state.socket of
-                        undefined ->
-                            {keep_state, State#state{streams = NStreams}};
-                        Socket ->
-                            Packet = encode_stream_frame(StreamId, {reset, {worker_down, Reason}}),
-                            ok = ssl:send(Socket, Packet),
-                            {keep_state, State#state{streams = NStreams}}
-                    end
-            end
-    end;
-
-%%% 处理内部消息。TLS 收到的数据先经过 protobuf 解码，再由这里按本地事件分发。
-
-%% 容器管理命令由 iot 发起，使用 command/command_response 语义。
-handle_event(internal, #'Command'{packet_id = PktId, body = {container, ContainerCommand}}, ?STATE_ACTIVATED, State = #state{socket = Socket}) ->
-    handle_container_command(PktId, ContainerCommand, Socket),
-    {keep_state, State};
-handle_event(internal, #'Command'{packet_id = PktId, body = {container, ContainerCommand}}, _StateName, State = #state{socket = Socket}) ->
-    logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [ContainerCommand]),
-    send_container_response(Socket, PktId, {error, <<"agent invalid">>}),
-    {keep_state, State};
-
-%% 处理response
-handle_event(internal, #'Response'{packet_id = AuthPktId, body = {auth_response, #'Response.AuthResponse'{}}}, ?STATE_AUTH, State = #state{auth_pkt_id = AuthPktId}) ->
-    logger:debug("[efka_iot_client] auth success"),
-    State1 = schedule_ssl_ping(State#state{auth_pkt_id = undefined}),
-    {next_state, ?STATE_ACTIVATED, State1, [{next_event, info, flush_cache}]};
-handle_event(internal, #'Response'{packet_id = AuthPktId, body = {error, #'Response.Error'{reason = Reason}}}, ?STATE_AUTH, State = #state{socket = Socket, auth_pkt_id = AuthPktId}) ->
-    logger:debug("[efka_iot_client] auth failed, reason: ~p", [Reason]),
-    disconnect(Socket),
-    schedule_reconnect(),
-    {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_pkt_id = undefined}};
-handle_event(internal, #'Response'{} = Reply, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore unexpected response in state ~p: ~p", [StateName, Reply]),
-    {keep_state, State};
-handle_event(internal, #'CommandResponse'{} = Reply, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore unexpected command_response in state ~p: ~p", [StateName, Reply]),
-    {keep_state, State};
-
-%% 透明 TCP stream 多路复用。
-handle_event(internal, #'Stream'{stream_id = StreamId, payload = Payload}, ?STATE_ACTIVATED, State) ->
-    Body = case Payload of
-               {open, #'Stream.Open'{target = Target}} ->
-                   {open, Target};
-               {opened, #'Stream.Opened'{}} ->
-                   opened;
-               {open_error, #'Stream.OpenError'{reason = Reason}} ->
-                   {open_error, Reason};
-               {data, #'Stream.Data'{bytes = Data}} ->
-                   {data, Data};
-               {fin, #'Stream.Fin'{}} ->
-                   fin;
-               {reset, #'Stream.Reset'{reason = Reason}} ->
-                   {reset, Reason}
-           end,
-    handle_stream_frame(StreamId, Body, State);
-handle_event(internal, #'Stream'{stream_id = StreamId, payload = Payload}, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore stream frame in state ~p, stream_id: ~p, body: ~p",
-        [StateName, StreamId, Payload]),
-    {keep_state, State};
-
-%% 处理Pub/Sub机制
-handle_event(internal, #'Message'{body = {pong, #'Message.Pong'{}}}, ?STATE_ACTIVATED, State) ->
-    {keep_state, State};
-handle_event(internal, #'Message'{body = {pub, #'Message.Pub'{topic = Topic, qos = Qos, content = Content}}}, ?STATE_ACTIVATED, State) ->
-    logger:debug("[efka_iot_client] get pub topic: ~p, qos: ~p, content: ~p", [Topic, Qos, Content]),
-    efka_subscription:publish(Topic, Qos, Content),
-    {keep_state, State};
-handle_event(internal, Packet, StateName, State) ->
-    logger:warning("[efka_iot_client] ignore unknown packet: ~p, state_name: ~p", [Packet, StateName]),
-    {keep_state, State};
-
-handle_event(info, Info, _, State = #state{}) ->
-    logger:notice("[efka_iot_client] get unknown info: ~p", [Info]),
-    {keep_state, State}.
-
--spec terminate(term(), atom(), #state{}) -> ok.
-terminate(Reason, _StateName, State = #state{socket = Socket, outbox = Outbox}) ->
-    cancel_ssl_ping(State),
-    _ = close_all_streams(Reason, State),
-    disconnect(Socket),
-    efka_iot_outbox:close(Outbox),
-    logger:notice("[efka_iot_client] terminate with reason: ~p", [Reason]),
-    ok.
-
--spec code_change(term(), atom(), #state{}, term()) -> {ok, atom(), #state{}}.
-code_change(_OldVsn, StateName, State = #state{}, _Extra) ->
-    {ok, StateName, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec auth_packet(pos_integer()) -> binary().
-auth_packet(PktId) when is_integer(PktId), PktId > 0 ->
-    {ok, AuthInfo} = application:get_env(efka, auth),
-    UUID = proplists:get_value(uuid, AuthInfo),
-    Token = proplists:get_value(token, AuthInfo),
-
-    Timestamp = efka_util:timestamp(),
-    encode_transport_frame(?CLASS_REQUEST, #'Request'{
-        packet_id = PktId,
-        body = {auth_request, #'Request.AuthRequest'{
-            uuid = list_to_binary(UUID),
-            token = list_to_binary(Token),
-            timestamp = Timestamp
-        }}
-    }).
-
--spec encode_message_frame(term()) -> binary().
-encode_message_frame(ping) ->
-    encode_transport_frame(?CLASS_MESSAGE, #'Message'{body = {ping, #'Message.Ping'{}}});
-encode_message_frame({metric_data, RouteKey, Metric}) ->
-    encode_transport_frame(?CLASS_MESSAGE, #'Message'{
-        body = {metric_data, #'Message.MetricData'{route_key = RouteKey, metric = Metric}}
-    });
-encode_message_frame(Body) ->
-    error({unsupported_message_body, Body}).
-
--spec encode_stream_frame(stream_id(), term()) -> binary().
-encode_stream_frame(StreamId, Body) ->
-    Payload = case Body of
-                  open ->
-                      {open, #'Stream.Open'{target = ?STREAM_TARGET_MANAGER}};
-                  {open, Target} when is_integer(Target), Target > 0 ->
-                      {open, #'Stream.Open'{target = Target}};
-                  opened ->
-                      {opened, #'Stream.Opened'{}};
-                  {open_error, Reason} ->
-                      {open_error, #'Stream.OpenError'{reason = reason_to_binary(Reason)}};
-                  {data, Data} when is_binary(Data) ->
-                      {data, #'Stream.Data'{bytes = Data}};
-                  fin ->
-                      {fin, #'Stream.Fin'{}};
-                  {reset, Reason} ->
-                      {reset, #'Stream.Reset'{reason = reason_to_binary(Reason)}}
-              end,
-    encode_transport_frame(?CLASS_STREAM, #'Stream'{stream_id = StreamId, payload = Payload}).
-
--spec encode_transport_frame(byte(), message_pb:'$msg'()) -> binary().
-encode_transport_frame(Class, Msg) ->
-    Payload = message_pb:encode_msg(Msg),
-    <<Class, Payload/binary>>.
-
--spec decode_frame(binary()) -> {ok, tuple()} | {error, term()}.
-decode_frame(<<?CLASS_RESPONSE, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Response');
-decode_frame(<<?CLASS_COMMAND, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Command');
-decode_frame(<<?CLASS_COMMAND_RESPONSE, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'CommandResponse');
-decode_frame(<<?CLASS_MESSAGE, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Message');
-decode_frame(<<?CLASS_STREAM, Payload/binary>>) ->
-    decode_pb_frame(Payload, 'Stream');
-decode_frame(<<?CLASS_REQUEST, _Payload/binary>>) ->
-    {error, unsupported_request_frame};
-decode_frame(Other) ->
-    {error, {invalid_frame, Other}}.
-
--spec decode_pb_frame(binary(), message_pb:'$msg_name'()) ->
-    {ok, tuple()} | {error, term()}.
-decode_pb_frame(Payload, MsgName) ->
-    try message_pb:decode_msg(Payload, MsgName) of
-        Msg ->
-            {ok, Msg}
-    catch
-        Class:Reason ->
-            {error, {bad_protobuf, MsgName, Class, Reason}}
-    end.
-
--spec connect_socket() -> {ok, ssl:sslsocket()} | {error, term()}.
-connect_socket() ->
-    {ok, Props} = application:get_env(efka, iot_server),
-    Host = proplists:get_value(host, Props),
-    Port = proplists:get_value(tls_port, Props),
-    SslOptions = [
-        binary,
-        {active, true},
-        {packet, 4},
-        {verify, verify_none}
-    ],
-    ssl:connect(Host, Port, SslOptions, 5000).
-
--spec disconnect(undefined | ssl:sslsocket()) -> ok.
-disconnect(undefined) ->
-    ok;
-disconnect(Socket) ->
-    catch ssl:close(Socket),
-    ok.
-
--spec schedule_reconnect() -> reference().
-schedule_reconnect() ->
-    erlang:start_timer(5000, self(), create_transport).
-
--spec schedule_ssl_ping(#state{}) -> #state{}.
-schedule_ssl_ping(State = #state{ping_timer_ref = TimerRef}) ->
-    cancel_timer(TimerRef),
-    State#state{ping_timer_ref = erlang:start_timer(?SSL_PING_INTERVAL, self(), ssl_ping)}.
-
--spec cancel_ssl_ping(#state{}) -> ok.
-cancel_ssl_ping(#state{ping_timer_ref = TimerRef}) ->
-    cancel_timer(TimerRef).
-
--spec cancel_timer(undefined | reference()) -> ok.
-cancel_timer(undefined) ->
-    ok;
-cancel_timer(TimerRef) ->
-    _ = erlang:cancel_timer(TimerRef),
-    ok.
-
--spec send_container_response(ssl:sslsocket(), pos_integer(), term()) -> ok.
-send_container_response(Socket, PktId, Reply) ->
-    Body = case Reply of
-               ok ->
-                   {result, <<"ok">>};
-               {ok, Result} ->
-                   {result, reply_to_binary(Result)};
-               {error, Reason} ->
-                   {error, #'CommandResponse.Error'{code = 1, reason = reason_to_binary(Reason)}};
-               Other ->
-                   {result, reply_to_binary(Other)}
-           end,
-    Packet = encode_transport_frame(?CLASS_COMMAND_RESPONSE, #'CommandResponse'{
-        packet_id = PktId,
-        body = Body
-    }),
-    ok = ssl:send(Socket, Packet).
-
--spec handle_container_command(pos_integer(), message_pb:'Command.Container'(), ssl:sslsocket()) -> ok.
-handle_container_command(PktId, #'Command.Container'{action = {list, #'Command.Container.ContainerList'{}}}, Socket) ->
-    Reply = docker_commands:get_containers(),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {start, #'Command.Container.ContainerStart'{target = Target}}}, Socket) ->
-    Reply = docker_commands:start_container(container_target(Target)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {stop, #'Command.Container.ContainerStop'{target = Target, timeout_seconds = TimeoutSeconds}}}, Socket) ->
-    Reply = docker_commands:stop_container(container_target(Target), TimeoutSeconds),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {kill, #'Command.Container.ContainerKill'{target = Target, signal = Signal}}}, Socket) ->
-    Reply = docker_commands:kill_container(container_target(Target), to_binary(Signal)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {remove, #'Command.Container.ContainerRemove'{target = Target, force = Force, remove_volumes = RemoveVolumes}}}, Socket) ->
-    Reply = docker_commands:remove_container(container_target(Target), to_bool(Force), to_bool(RemoveVolumes)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, #'Command.Container'{action = {config, #'Command.Container.ContainerConfig'{target = Target, config = Config}}}, Socket) ->
-    Reply = docker_helper:update_container_config(container_target(Target), iolist_to_binary(Config)),
-    send_container_response(Socket, PktId, Reply),
-    ok;
-handle_container_command(PktId, ContainerCommand, Socket) ->
-    logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [ContainerCommand]),
-    send_container_response(Socket, PktId, {error, <<"agent invalid">>}),
-    ok.
-
-%% iot 发起 open 时携带 target；efka 根据本地 stream_targets 配置建连。
--spec handle_stream_frame(term(), term(), #state{}) -> gen_statem:event_handler_result(atom(), #state{}).
-handle_stream_frame(StreamId, {open, Target}, State = #state{streams = Streams})
-    when is_integer(StreamId), StreamId > 0, is_integer(Target), Target > 0 ->
-    case valid_iot_stream_id(StreamId) andalso not maps:is_key(StreamId, Streams) of
-        true ->
-            case start_stream_worker(StreamId, Target) of
-                {ok, {WorkerPid, MonitorRef}} ->
-                    StreamState = #stream_state{worker_pid = WorkerPid, monitor_ref = MonitorRef},
-                    {keep_state, State#state{streams = maps:put(StreamId, StreamState, Streams)}};
-                {error, Reason} ->
-                    send_stream(StreamId, {open_error, Reason}),
-                    {keep_state, State}
-            end;
-        false ->
-            send_stream(StreamId, {reset, <<"invalid stream open">>}),
-            {keep_state, State}
-    end;
-handle_stream_frame(StreamId, {open, Target}, State)
-    when is_integer(StreamId), StreamId > 0 ->
-    logger:warning("[efka_iot_client] invalid stream target, stream_id: ~p, target: ~p", [StreamId, Target]),
-    send_stream(StreamId, {open_error, <<"invalid stream target">>}),
-    {keep_state, State};
-handle_stream_frame(StreamId, Body, State = #state{streams = Streams})
-    when is_integer(StreamId), StreamId > 0 ->
-    case maps:get(StreamId, Streams, undefined) of
-        undefined ->
-            maybe_reset_unknown_stream(StreamId, Body),
-            {keep_state, State};
-        StreamState = #stream_state{worker_pid = WorkerPid} ->
-            WorkerPid ! {stream, StreamId, Body},
-            case Body of
-                {reset, _Reason} ->
-                    demonitor_stream(StreamState),
-                    {keep_state, State#state{streams = maps:remove(StreamId, Streams)}};
-                _ ->
-                    {keep_state, State}
-            end
-    end;
-handle_stream_frame(StreamId, Body, State) ->
-    logger:warning("[efka_iot_client] invalid stream frame, stream_id: ~p, body: ~p", [StreamId, Body]),
-    {keep_state, State}.
-
--spec start_stream_worker(stream_id(), pos_integer()) -> {ok, {pid(), reference()}} | {error, term()}.
-start_stream_worker(StreamId, ?STREAM_TARGET_MANAGER) ->
-    efka_iot_stream:start_stream(StreamId, ?STREAM_TARGET_MANAGER);
-start_stream_worker(StreamId, ?STREAM_TARGET_CONTAINER_DEPLOY) ->
-    efka_iot_deploy_stream:start_stream(StreamId);
-start_stream_worker(_StreamId, Target) ->
-    {error, {unknown_stream_target, Target}}.
-
--spec maybe_reset_unknown_stream(stream_id(), term()) -> ok.
-maybe_reset_unknown_stream(_StreamId, {reset, _Reason}) ->
-    ok;
-maybe_reset_unknown_stream(_StreamId, fin) ->
-    ok;
-maybe_reset_unknown_stream(_StreamId, {open_error, _Reason}) ->
-    ok;
-maybe_reset_unknown_stream(StreamId, _Body) ->
-    send_stream(StreamId, {reset, <<"unknown stream">>}).
-
--spec valid_iot_stream_id(stream_id()) -> boolean().
-valid_iot_stream_id(StreamId) ->
-    StreamId rem 2 =:= 1.
-
--spec take_stream_by_monitor(reference(), pid(), map()) ->
-    {stream_id(), #stream_state{}, map()} | error.
-take_stream_by_monitor(MonitorRef, WorkerPid, Streams) ->
-    take_stream_by_monitor(MonitorRef, WorkerPid, maps:iterator(Streams), Streams).
-
-take_stream_by_monitor(MonitorRef, WorkerPid, Iter, Streams) ->
-    case maps:next(Iter) of
-        none ->
-            error;
-        {StreamId, StreamState = #stream_state{worker_pid = WorkerPid, monitor_ref = MonitorRef}, _NextIter} ->
-            {StreamId, StreamState, maps:remove(StreamId, Streams)};
-        {_StreamId, _StreamState, NextIter} ->
-            take_stream_by_monitor(MonitorRef, WorkerPid, NextIter, Streams)
-    end.
-
--spec close_all_streams(term(), #state{}) -> #state{}.
-close_all_streams(Reason, State = #state{streams = Streams}) ->
-    maps:foreach(fun(StreamId, StreamState = #stream_state{worker_pid = WorkerPid}) ->
-        demonitor_stream(StreamState),
-        WorkerPid ! {stream, StreamId, {reset, {channel_closed, Reason}}}
-    end, Streams),
-    State#state{streams = #{}}.
-
--spec demonitor_stream(#stream_state{}) -> ok.
-demonitor_stream(#stream_state{monitor_ref = MonitorRef}) ->
-    erlang:demonitor(MonitorRef, [flush]),
-    ok.
-
--spec reply_to_binary(term()) -> binary().
-reply_to_binary(Value) when is_binary(Value) ->
-    Value;
-reply_to_binary(Value) ->
-    try iolist_to_binary(Value) of
-        Bin ->
-            Bin
-    catch
-        _:_ ->
-            try iolist_to_binary(json:encode(Value)) of
-                JsonBin ->
-                    JsonBin
-            catch
-                _:_ ->
-                    unicode:characters_to_binary(io_lib:format("~p", [Value]))
-            end
-    end.
-
--spec reason_to_binary(term()) -> binary().
-reason_to_binary(Reason) when is_binary(Reason) ->
-    Reason;
-reason_to_binary(Reason) ->
-    try iolist_to_binary(Reason) of
-        Bin ->
-            Bin
-    catch
-        _:_ ->
-            unicode:characters_to_binary(io_lib:format("~p", [Reason]))
-    end.
-
--spec container_target(message_pb:'Command.Container.ContainerTarget'() | undefined) -> binary().
-container_target(#'Command.Container.ContainerTarget'{name = Name, id = Id}) ->
-    NameBin = iolist_to_binary(Name),
-    IdBin = iolist_to_binary(Id),
-    case NameBin of
-        <<>> ->
-            true = IdBin =/= <<>>,
-            IdBin;
-        _ ->
-            NameBin
-    end;
-container_target(undefined) ->
-    error(bad_container_target).
-
--spec to_binary(binary() | list()) -> binary().
-to_binary(Value) when is_binary(Value) ->
-    Value;
-to_binary(Value) when is_list(Value) ->
-    unicode:characters_to_binary(Value).
-
--spec to_bool(true | false | 0 | 1) -> boolean().
-to_bool(true) ->
-    true;
-to_bool(1) ->
-    true;
-to_bool(false) ->
-    false;
-to_bool(0) ->
-    false.

apps/efka/src/iot/efka_iot_deploy_stream.erl

@@ -1,119 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc One container deploy feedback stream from iot.
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_deploy_stream).
-
--export([start_stream/1]).
--export([run/1]).
-
--define(REQUEST_TIMEOUT, 10000).
--define(MAX_REQUEST_BYTES, 16 * 1024 * 1024).
-
--type stream_id() :: pos_integer().
-
--spec start_stream(StreamId :: stream_id()) -> {ok, {pid(), reference()}}.
-start_stream(StreamId) when is_integer(StreamId), StreamId > 0 ->
-    {ok, spawn_monitor(?MODULE, run, [StreamId])}.
-
--spec run(StreamId :: stream_id()) -> ok.
-run(StreamId) when is_integer(StreamId), StreamId > 0 ->
-    try run0(StreamId) of
-        ok ->
-            ok
-    catch
-        Class:Reason:Stack ->
-            logger:warning("[efka_iot_deploy_stream] stream_id: ~p crashed, class: ~p, reason: ~p, stack: ~p",
-                [StreamId, Class, Reason, Stack]),
-            send_error_and_close(StreamId, iolist_to_binary(io_lib:format("~p:~p", [Class, Reason]))),
-            ok
-    after
-        efka_iot_client:stream_done(StreamId)
-    end.
-
--spec run0(stream_id()) -> ok.
-run0(StreamId) ->
-    efka_iot_client:send_stream(StreamId, opened),
-    case receive_request(StreamId, <<>>) of
-        {ok, Request} ->
-            handle_request(StreamId, Request);
-        {error, reset} ->
-            ok;
-        {error, Reason} ->
-            send_error_and_close(StreamId, reason_to_binary(Reason))
-    end.
-
--spec receive_request(stream_id(), binary()) -> {ok, binary()} | {error, term()}.
-receive_request(StreamId, Acc) ->
-    receive
-        {stream, StreamId, {data, Data}} when is_binary(Data) ->
-            NAcc = <<Acc/binary, Data/binary>>,
-            case byte_size(NAcc) =< ?MAX_REQUEST_BYTES of
-                true ->
-                    receive_request(StreamId, NAcc);
-                false ->
-                    {error, request_too_large}
-            end;
-        {stream, StreamId, fin} ->
-            {ok, Acc};
-        {stream, StreamId, {reset, _Reason}} ->
-            {error, reset};
-        Info ->
-            logger:debug("[efka_iot_deploy_stream] stream_id: ~p ignore unknown info: ~p", [StreamId, Info]),
-            receive_request(StreamId, Acc)
-    after ?REQUEST_TIMEOUT ->
-        {error, request_timeout}
-    end.
-
--spec handle_request(stream_id(), binary()) -> ok.
-handle_request(StreamId, RequestBin) ->
-    case decode_request(RequestBin) of
-        {ok, TaskId, Params} ->
-            deploy(StreamId, TaskId, Params);
-        {error, Reason} ->
-            send_error_and_close(StreamId, reason_to_binary(Reason))
-    end.
-
--spec decode_request(binary()) -> {ok, non_neg_integer(), map()} | {error, term()}.
-decode_request(RequestBin) ->
-    try json:decode(RequestBin) of
-        #{<<"task_id">> := TaskId, <<"params">> := Params}
-          when is_integer(TaskId), TaskId >= 0, is_map(Params) ->
-            {ok, TaskId, Params};
-        _ ->
-            {error, invalid_deploy_request}
-    catch
-        Class:Reason ->
-            {error, {bad_json, Class, Reason}}
-    end.
-
--spec deploy(stream_id(), non_neg_integer(), map()) -> ok.
-deploy(StreamId, TaskId, Params) ->
-    try
-        ContainerName = maps:get(<<"container_name">>, Params),
-        {ok, RootDir} = docker_helper:root_dir(),
-        {ok, ContainerDir} = docker_helper:ensure_container_dir(RootDir, ContainerName),
-        docker_deployer:deploy(TaskId, ContainerDir, Params, {stream, StreamId})
-    catch
-        Class:Reason ->
-            Error = iolist_to_binary(io_lib:format("deploy stream failed: ~p:~p", [Class, Reason])),
-            send_error_and_close(StreamId, Error)
-    end.
-
--spec send_error_and_close(stream_id(), binary()) -> ok.
-send_error_and_close(StreamId, Reason) ->
-    ok = efka_iot_client:send_stream(StreamId, {data, iolist_to_binary(json:encode(#{
-        <<"type">> => <<"error">>,
-        <<"message">> => Reason
-    }))}),
-    ok = efka_iot_client:send_stream(StreamId, {data, iolist_to_binary(json:encode(#{
-        <<"type">> => <<"close">>,
-        <<"reason">> => <<"fail">>
-    }))}),
-    efka_iot_client:send_stream(StreamId, fin).
-
--spec reason_to_binary(term()) -> binary().
-reason_to_binary(Reason) when is_binary(Reason) ->
-    Reason;
-reason_to_binary(Reason) ->
-    unicode:characters_to_binary(io_lib:format("~p", [Reason])).

apps/efka/src/iot/efka_iot_heartbeat.erl

@@ -1,119 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc UDP heartbeat sender for iot host liveness.
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_heartbeat).
-
--behaviour(gen_server).
-
-%% API
--export([start_link/0]).
-
-%% gen_server callbacks
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
-
--define(SERVER, ?MODULE).
--define(HEARTBEAT_VERSION, 1).
--define(HEARTBEAT_NONCE_BYTES, 16).
--define(DEFAULT_INTERVAL, 5000).
-
--record(state, {
-    socket :: gen_udp:socket(),
-    host :: inet:hostname() | inet:ip_address(),
-    port :: inet:port_number(),
-    interval :: pos_integer(),
-    uuid :: binary(),
-    heartbeat_secret :: binary()
-}).
-
-%%%===================================================================
-%%% API
-%%%===================================================================
-
--spec start_link() -> {ok, pid()} | ignore | {error, term()}.
-start_link() ->
-    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
-
-%%%===================================================================
-%%% gen_server callbacks
-%%%===================================================================
-
--spec init(list()) -> {ok, #state{}} | {stop, term()}.
-init([]) ->
-    ok = application:ensure_started(crypto),
-    {ok, ServerProps} = application:get_env(efka, iot_server),
-    {ok, AuthProps} = application:get_env(efka, auth),
-    HeartbeatProps = case application:get_env(efka, heartbeat) of
-        {ok, Props} -> Props;
-        undefined -> []
-    end,
-    Host = proplists:get_value(host, ServerProps),
-    UdpPort = proplists:get_value(udp_port, ServerProps),
-    Interval = proplists:get_value(interval, HeartbeatProps, ?DEFAULT_INTERVAL),
-    UUID = list_to_binary(proplists:get_value(uuid, AuthProps)),
-    Token = list_to_binary(proplists:get_value(token, AuthProps)),
-    HeartbeatSecret = crypto:hash(sha256, Token),
-    case gen_udp:open(0, [binary]) of
-        {ok, Socket} ->
-            erlang:send_after(0, self(), heartbeat),
-            {ok, #state{
-                socket = Socket,
-                host = Host,
-                port = UdpPort,
-                interval = Interval,
-                uuid = UUID,
-                heartbeat_secret = HeartbeatSecret
-            }};
-        {error, Reason} ->
-            {stop, Reason}
-    end.
-
--spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, ok, #state{}}.
-handle_call(_Request, _From, State) ->
-    {reply, ok, State}.
-
--spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
-handle_cast(_Request, State) ->
-    {noreply, State}.
-
--spec handle_info(term(), #state{}) -> {noreply, #state{}}.
-handle_info(heartbeat, State = #state{interval = Interval}) ->
-    ok = send_heartbeat(State),
-    erlang:send_after(Interval, self(), heartbeat),
-    {noreply, State};
-handle_info(Info, State) ->
-    logger:warning("[efka_iot_heartbeat] ignore unknown info: ~p", [Info]),
-    {noreply, State}.
-
--spec terminate(term(), #state{}) -> ok.
-terminate(_Reason, #state{socket = Socket}) ->
-    gen_udp:close(Socket),
-    ok.
-
--spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
-code_change(_OldVsn, State, _Extra) ->
-    {ok, State}.
-
-%%%===================================================================
-%%% Internal functions
-%%%===================================================================
-
--spec send_heartbeat(#state{}) -> ok.
-send_heartbeat(#state{socket = Socket, host = Host, port = Port, uuid = UUID, heartbeat_secret = HeartbeatSecret}) ->
-    Packet = heartbeat_packet_iodata(UUID, HeartbeatSecret),
-    case gen_udp:send(Socket, Host, Port, Packet) of
-        ok ->
-            ok;
-        {error, Reason} ->
-            logger:warning("[efka_iot_heartbeat] send heartbeat failed, reason: ~p", [Reason]),
-            ok
-    end.
-
--spec heartbeat_packet_iodata(binary(), binary()) -> iodata().
-heartbeat_packet_iodata(UUID, HeartbeatSecret) when is_binary(UUID), is_binary(HeartbeatSecret) ->
-    UUIDLen = byte_size(UUID),
-    Timestamp = efka_util:timestamp(),
-    Nonce = crypto:strong_rand_bytes(?HEARTBEAT_NONCE_BYTES),
-    Payload = <<?HEARTBEAT_VERSION:8, UUIDLen:16, UUID:UUIDLen/binary, Timestamp:64/unsigned-big, Nonce/binary>>,
-    Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload),
-    [Payload, Mac].

apps/efka/src/iot/efka_iot_outbox.erl

@@ -1,662 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc
-%%% efka_iot_client 的持久化待发送队列。
-%%%
-%%% 本模块用于保存 efka_iot_client 离线期间产生的待发送 packet。底层使用
-%%% 追加写 segment 文件的方式实现。当前场景只有一个写入方和一个读取方，并且
-%%% 读取方每次只读取一条数据。
-%%%
-%%% 文件结构：
-%%% - `segment-N.log'：一个 segment 文件，按写入顺序追加保存 record。
-%%% - `metadata.json'：使用原始 JSON 文本保存 write_seq、acked_seq 和
-%%%   writer_segment。进程启动时仍然会扫描 segment 文件，并以 segment 文件
-%%%   里的真实数据作为准确信息；因此即使 metadata 落后，也不会导致跳过或删除
-%%%   未消费的数据。
-%%% - outbox 目录通过 open/1 的 dir 参数传入，本模块不直接读取应用环境变量。
-%%%
-%%% open/1 参数：
-%%% - dir：outbox 数据目录。
-%%% - segment_record_limit：每个 segment 最多保存的 record 数。
-%%% - max_segments：最多保留的 live segment 数。
-%%% - max_record_bytes：允许读取的单条 record 最大 payload 字节数。
-%%%
-%%% record 格式：
-%%% - 8 字节 magic bytes，当前为 <<"EFKAOBX1">>。
-%%% - 1 字节 version，当前为 1。
-%%% - 1 字节 header size，当前为 22。
-%%% - 4 字节 unsigned big-endian packet size。
-%%% - 8 字节 unsigned big-endian Seq。
-%%% - Packet 原始二进制内容。
-%%% - 4 字节 unsigned big-endian crc32，校验范围为 header + Packet。
-%%% - 读取时会先搜索 magic，再校验 version、header size、packet size 和
-%%%   crc32。遇到损坏 record 时，会继续向后寻找下一个合法 magic，尽量恢复
-%%%   后续可读数据。
-%%%
-%%% 写入逻辑：
-%%% - `append/2' 将一条 record 写入当前 writer segment，随后 fsync 当前
-%%%   segment，推进 write_seq，并持久化 metadata。
-%%% - 每个 segment 最多保存多少条数据由 open/1 的 segment_record_limit
-%%%   参数决定。
-%%% - outbox 最多保留多少个 live segment 由 open/1 的 max_segments 参数
-%%%   决定。当当前 segment 已满，且 live segment 数已经达到上限时，新
-%%%   packet 会被丢弃，已有磁盘数据保持不变。
-%%% - 当当前 segment 已满，但 live segment 数量尚未达到上限时，writer 会
-%%%   关闭旧文件，并开始写入 `segment-(N + 1).log'。
-%%%
-%%% 读取逻辑：
-%%% - `next/1' 找到第一个 end_seq 大于 acked_seq 的 live segment，打开该
-%%%   segment，并只返回第一条 Seq 大于 acked_seq 的 record。
-%%% - 本模块不做批量读取；efka_iot_client 每次只发送并 ack 一条 packet。
-%%% - `ack/2' 接受当前读取到的 Seq。正常情况下 Seq 连续递增；如果某条
-%%%   record 损坏并被读取端跳过，ack 允许推进到后续合法 record 的 Seq，
-%%%   避免单条坏数据阻断整个 segment 后续数据的消费。
-%%%
-%%% segment 清理逻辑：
-%%% - 当 `ack/2' 消费完某个 segment 的最后一条 record，并且后续 segment
-%%%   里还有未读数据时，已消费的 segment 文件会被删除；下一次读取会从后续
-%%%   segment 开始。
-%%% - 当 acked_seq 追上 write_seq 时，说明所有 record 都已经投递完成；此时
-%%%   会删除所有 segment 文件，将计数器重置为 0，下一次 append 从全新的
-%%%   `segment-1.log' 的 Seq 1 开始。
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_outbox).
-
--export([open/1, close/1, append/2, next/1, ack/2]).
--export_type([outbox/0]).
-
--record(segment, {
-    id :: pos_integer(),
-    path :: file:filename_all(),
-    start_seq = 0 :: non_neg_integer(),
-    end_seq = 0 :: non_neg_integer(),
-    records = 0 :: non_neg_integer()
-}).
-
--record(outbox, {
-    dir :: file:filename_all(),
-    metadata_path :: file:filename_all(),
-    writer_segment = 1 :: pos_integer(),
-    fd :: file:fd(),
-    segment_record_limit :: pos_integer(),
-    max_segments :: pos_integer(),
-    max_record_bytes :: pos_integer(),
-    segments = [] :: [#segment{}],
-    next_seq = 1 :: pos_integer(),
-    write_seq = 0 :: non_neg_integer(),
-    acked_seq = 0 :: non_neg_integer()
-}).
-
--type outbox() :: #outbox{}.
-
--define(METADATA_FILE, "metadata.json").
--define(SEGMENT_PREFIX, "segment-").
--define(SEGMENT_EXT, ".log").
--define(RECORD_MAGIC, <<"EFKAOBX1">>).
--define(RECORD_MAGIC_SIZE, 8).
--define(RECORD_VERSION, 1).
--define(RECORD_HEADER_SIZE, 22).
--define(RECORD_CRC_SIZE, 4).
-
--type open_options() :: #{
-    dir := file:filename_all(),
-    segment_record_limit := pos_integer(),
-    max_segments := pos_integer(),
-    max_record_bytes := pos_integer()
-}.
-
--spec open(open_options()) -> {ok, outbox()} | {error, term()}.
-open(Options) when is_map(Options) ->
-    Dir = maps:get(dir, Options),
-    SegmentRecordLimit = positive_option(segment_record_limit, Options),
-    MaxSegments = positive_option(max_segments, Options),
-    MaxRecordBytes = positive_option(max_record_bytes, Options),
-    MetadataPath = filename:join(Dir, ?METADATA_FILE),
-    maybe
-        ok ?= ensure_dir(Dir),
-        {_MetaWriteSeq, MetaAckedSeq, MetaWriterSegment} = read_metadata(MetadataPath),
-        {ok, Segments} ?= load_segments(Dir, MaxRecordBytes),
-        ScannedWriteSeq = max_segment_end(Segments),
-        WriteSeq = ScannedWriteSeq,
-        AckedSeq = min(MetaAckedSeq, WriteSeq),
-        WriterSegment = writer_segment_id(Segments, MetaWriterSegment),
-        {ok, Fd} ?= open_writer(segment_path(Dir, WriterSegment)),
-        Outbox0 = #outbox{
-            dir = Dir,
-            metadata_path = MetadataPath,
-            writer_segment = WriterSegment,
-            fd = Fd,
-            segment_record_limit = SegmentRecordLimit,
-            max_segments = MaxSegments,
-            max_record_bytes = MaxRecordBytes,
-            segments = Segments,
-            next_seq = WriteSeq + 1,
-            write_seq = WriteSeq,
-            acked_seq = AckedSeq
-        },
-        {ok, Outbox1} ?= normalize_open_outbox(Outbox0),
-        ok ?= persist_metadata(Outbox1),
-        {ok, Outbox1}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec close(outbox()) -> ok.
-close(#outbox{fd = Fd}) ->
-    _ = file:close(Fd),
-    ok.
-
--spec append(binary(), outbox()) ->
-    {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}.
-append(Packet, Outbox0) when is_binary(Packet) ->
-    case ensure_writable_segment(Outbox0) of
-        {ok, Outbox1 = #outbox{fd = Fd, next_seq = Seq}} ->
-            Record = encode_record(Seq, Packet),
-            maybe
-                ok ?= file:write(Fd, Record),
-                ok ?= file:sync(Fd),
-                Outbox2 = mark_written(Outbox1, Seq),
-                ok ?= persist_metadata(Outbox2),
-                {ok, Outbox2}
-            else
-                {error, Reason} ->
-                    {error, Reason}
-            end;
-        {dropped, capacity_reached, Outbox1} ->
-            {dropped, capacity_reached, Outbox1};
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec next(outbox()) -> eof | {ok, pos_integer(), binary()} | {error, term()}.
-next(#outbox{segments = Segments, acked_seq = AckedSeq, max_record_bytes = MaxRecordBytes}) ->
-    case next_segment(Segments, AckedSeq) of
-        undefined ->
-            eof;
-        #segment{path = Path} ->
-            case file:open(Path, [read, binary]) of
-                {ok, Fd} ->
-                    Result = read_next_unacked(Fd, AckedSeq, MaxRecordBytes),
-                    _ = file:close(Fd),
-                    Result;
-                {error, enoent} ->
-                    eof;
-                {error, Reason} ->
-                    {error, Reason}
-            end
-    end.
-
--spec ack(pos_integer(), outbox()) -> {ok, outbox()} | {error, term()}.
-ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq =< AckedSeq ->
-    {ok, Outbox};
-ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq > AckedSeq ->
-    AckedOutbox = Outbox#outbox{acked_seq = Seq},
-    maybe
-        ok ?= persist_metadata(AckedOutbox),
-        {ok, NOutbox} ?= maybe_reset_or_prune(AckedOutbox),
-        ok ?= persist_metadata(NOutbox),
-        {ok, NOutbox}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end;
-ack(Seq, #outbox{acked_seq = AckedSeq}) when is_integer(Seq) ->
-    {error, {invalid_ack, Seq, AckedSeq}}.
-
--spec ensure_dir(file:filename_all()) -> ok | {error, term()}.
-ensure_dir(Dir) ->
-    filelib:ensure_dir(filename:join(Dir, "dummy")).
-
--spec positive_option(atom(), map()) -> pos_integer().
-positive_option(Key, Options) ->
-    case maps:get(Key, Options) of
-        Value when is_integer(Value), Value > 0 ->
-            Value;
-        Value ->
-            error({invalid_option, Key, Value})
-    end.
-
--spec read_metadata(file:filename_all()) ->
-    {non_neg_integer(), non_neg_integer(), pos_integer()}.
-read_metadata(MetadataPath) ->
-    case file:read_file(MetadataPath) of
-        {ok, Bin} ->
-            decode_metadata(Bin);
-        {error, _} ->
-            {0, 0, 1}
-    end.
-
--spec decode_metadata(binary()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}.
-decode_metadata(Bin) ->
-    try json:decode(Bin) of
-        Metadata ->
-            safe_metadata(Metadata)
-    catch
-        _:_Reason ->
-            {0, 0, 1}
-    end.
-
--spec safe_metadata(term()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}.
-safe_metadata(#{
-    <<"write_seq">> := WriteSeq,
-    <<"acked_seq">> := AckedSeq,
-    <<"writer_segment">> := WriterSegment
-})
-  when is_integer(WriteSeq), WriteSeq >= 0,
-       is_integer(AckedSeq), AckedSeq >= 0,
-       is_integer(WriterSegment), WriterSegment > 0 ->
-    {WriteSeq, AckedSeq, WriterSegment};
-safe_metadata(#{<<"write_seq">> := WriteSeq, <<"acked_seq">> := AckedSeq})
-  when is_integer(WriteSeq), WriteSeq >= 0, is_integer(AckedSeq), AckedSeq >= 0 ->
-    {WriteSeq, AckedSeq, 1};
-safe_metadata(_) ->
-    {0, 0, 1}.
-
--spec persist_metadata(outbox()) -> ok | {error, term()}.
-persist_metadata(#outbox{
-    metadata_path = MetadataPath,
-    writer_segment = WriterSegment,
-    write_seq = WriteSeq,
-    acked_seq = AckedSeq
-}) ->
-    Metadata = iolist_to_binary(json:encode(#{
-        <<"write_seq">> => WriteSeq,
-        <<"acked_seq">> => AckedSeq,
-        <<"writer_segment">> => WriterSegment
-    })),
-    TmpPath = MetadataPath ++ ".tmp",
-    case file:write_file(TmpPath, Metadata, [write, binary]) of
-        ok ->
-            file:rename(TmpPath, MetadataPath);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec load_segments(file:filename_all(), pos_integer()) -> {ok, [#segment{}]} | {error, term()}.
-load_segments(Dir, MaxRecordBytes) ->
-    case file:list_dir(Dir) of
-        {ok, Names} ->
-            Ids = lists:sort([Id || Name <- Names, {ok, Id} <- [parse_segment_id(Name)]]),
-            load_segments(Dir, Ids, MaxRecordBytes, []);
-        {error, enoent} ->
-            {ok, []};
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec load_segments(file:filename_all(), [pos_integer()], pos_integer(), [#segment{}]) ->
-    {ok, [#segment{}]} | {error, term()}.
-load_segments(_Dir, [], _MaxRecordBytes, Acc) ->
-    {ok, lists:reverse(Acc)};
-load_segments(Dir, [Id | Rest], MaxRecordBytes, Acc) ->
-    Path = segment_path(Dir, Id),
-    case scan_segment(Path, Id, MaxRecordBytes) of
-        {ok, undefined} ->
-            load_segments(Dir, Rest, MaxRecordBytes, Acc);
-        {ok, Segment} ->
-            load_segments(Dir, Rest, MaxRecordBytes, [Segment | Acc]);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec parse_segment_id(string()) -> {ok, pos_integer()} | error.
-parse_segment_id(Name) ->
-    case re:run(Name, "^" ++ ?SEGMENT_PREFIX ++ "([0-9]+)\\" ++ ?SEGMENT_EXT ++ "$",
-        [{capture, [1], list}])
-    of
-        {match, [Digits]} ->
-            case list_to_integer(Digits) of
-                Id when Id > 0 ->
-                    {ok, Id};
-                _ ->
-                    error
-            end;
-        nomatch ->
-            error
-    end.
-
--spec segment_path(file:filename_all(), pos_integer()) -> file:filename_all().
-segment_path(Dir, Id) ->
-    filename:join(Dir, ?SEGMENT_PREFIX ++ integer_to_list(Id) ++ ?SEGMENT_EXT).
-
--spec open_writer(file:filename_all()) -> {ok, file:fd()} | {error, term()}.
-open_writer(Path) ->
-    file:open(Path, [append, binary]).
-
--spec writer_segment_id([#segment{}], pos_integer()) -> pos_integer().
-writer_segment_id([], MetaWriterSegment) ->
-    max(1, MetaWriterSegment);
-writer_segment_id(Segments, _MetaWriterSegment) ->
-    (lists:last(Segments))#segment.id.
-
--spec max_segment_end([#segment{}]) -> non_neg_integer().
-max_segment_end([]) ->
-    0;
-max_segment_end(Segments) ->
-    lists:max([Segment#segment.end_seq || Segment <- Segments]).
-
--spec normalize_open_outbox(outbox()) -> {ok, outbox()} | {error, term()}.
-normalize_open_outbox(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) when Seq > 0 ->
-    reset_empty_outbox(Outbox);
-normalize_open_outbox(Outbox) ->
-    prune_acked_segments(Outbox).
-
--spec ensure_writable_segment(outbox()) ->
-    {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}.
-ensure_writable_segment(Outbox = #outbox{segments = Segments, writer_segment = WriterSegment}) ->
-    case find_segment(WriterSegment, Segments) of
-        undefined ->
-            {ok, Outbox};
-        #segment{records = Records} when Records < Outbox#outbox.segment_record_limit ->
-            {ok, Outbox};
-        #segment{} when length(Segments) >= Outbox#outbox.max_segments ->
-            {dropped, capacity_reached, Outbox};
-        #segment{} ->
-            rotate_writer(Outbox, WriterSegment + 1)
-    end.
-
--spec rotate_writer(outbox(), pos_integer()) -> {ok, outbox()} | {error, term()}.
-rotate_writer(Outbox = #outbox{dir = Dir, fd = Fd}, NewSegment) ->
-    maybe
-        ok ?= file:sync(Fd),
-        {ok, NFd} ?= open_writer(segment_path(Dir, NewSegment)),
-        _ = file:close(Fd),
-        {ok, Outbox#outbox{writer_segment = NewSegment, fd = NFd}}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec mark_written(outbox(), pos_integer()) -> outbox().
-mark_written(Outbox = #outbox{
-    dir = Dir,
-    writer_segment = WriterSegment,
-    segments = Segments
-}, Seq) ->
-    Path = segment_path(Dir, WriterSegment),
-    Segment0 = case find_segment(WriterSegment, Segments) of
-        undefined ->
-            #segment{id = WriterSegment, path = Path, start_seq = Seq};
-        Segment ->
-            Segment
-    end,
-    Records = Segment0#segment.records + 1,
-    Segment1 = Segment0#segment{end_seq = Seq, records = Records},
-    Outbox#outbox{
-        segments = replace_segment(Segment1, Segments),
-        next_seq = Seq + 1,
-        write_seq = Seq
-    }.
-
--spec replace_segment(#segment{}, [#segment{}]) -> [#segment{}].
-replace_segment(Segment, Segments) ->
-    lists:sort(
-        fun(A, B) -> A#segment.id < B#segment.id end,
-        [Segment | [S || S <- Segments, S#segment.id =/= Segment#segment.id]]
-    ).
-
--spec find_segment(pos_integer(), [#segment{}]) -> #segment{} | undefined.
-find_segment(Id, Segments) ->
-    case [Segment || Segment <- Segments, Segment#segment.id =:= Id] of
-        [Segment] ->
-            Segment;
-        [] ->
-            undefined
-    end.
-
--spec next_segment([#segment{}], non_neg_integer()) -> #segment{} | undefined.
-next_segment([], _AckedSeq) ->
-    undefined;
-next_segment([Segment = #segment{end_seq = EndSeq} | _Rest], AckedSeq) when EndSeq > AckedSeq ->
-    Segment;
-next_segment([_Segment | Rest], AckedSeq) ->
-    next_segment(Rest, AckedSeq).
-
--spec maybe_reset_or_prune(outbox()) -> {ok, outbox()} | {error, term()}.
-maybe_reset_or_prune(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) ->
-    reset_empty_outbox(Outbox);
-maybe_reset_or_prune(Outbox) ->
-    prune_acked_segments(Outbox).
-
--spec prune_acked_segments(outbox()) -> {ok, outbox()} | {error, term()}.
-prune_acked_segments(Outbox = #outbox{segments = Segments, acked_seq = AckedSeq}) ->
-    {DeleteSegments, KeepSegments} = take_acked_segments(Segments, AckedSeq, []),
-    maybe
-        ok ?= delete_files([Segment#segment.path || Segment <- DeleteSegments]),
-        {ok, Outbox#outbox{segments = KeepSegments}}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec take_acked_segments([#segment{}], non_neg_integer(), [#segment{}]) ->
-    {[#segment{}], [#segment{}]}.
-take_acked_segments([Segment = #segment{end_seq = EndSeq} | Rest], AckedSeq, Acc)
-  when EndSeq =< AckedSeq ->
-    take_acked_segments(Rest, AckedSeq, [Segment | Acc]);
-take_acked_segments(Segments, _AckedSeq, Acc) ->
-    {lists:reverse(Acc), Segments}.
-
--spec reset_empty_outbox(outbox()) -> {ok, outbox()} | {error, term()}.
-reset_empty_outbox(Outbox = #outbox{
-    dir = Dir,
-    fd = Fd,
-    writer_segment = WriterSegment,
-    segments = Segments
-}) ->
-    CurrentWriterPath = segment_path(Dir, WriterSegment),
-    SegmentPaths = [Segment#segment.path || Segment <- Segments],
-    Paths = lists:usort([CurrentWriterPath | SegmentPaths]),
-    maybe
-        ok ?= file:close(Fd),
-        ok ?= delete_files(Paths),
-        {ok, NFd} ?= open_writer(segment_path(Dir, 1)),
-        {ok, Outbox#outbox{
-            writer_segment = 1,
-            fd = NFd,
-            segments = [],
-            next_seq = 1,
-            write_seq = 0,
-            acked_seq = 0
-        }}
-    else
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec delete_files([file:filename_all()]) -> ok | {error, term()}.
-delete_files([]) ->
-    ok;
-delete_files([Path | Rest]) ->
-    case file:delete(Path) of
-        ok ->
-            delete_files(Rest);
-        {error, enoent} ->
-            delete_files(Rest);
-        {error, Reason} ->
-            {error, {delete_failed, Path, Reason}}
-    end.
-
--spec scan_segment(file:filename_all(), pos_integer(), pos_integer()) ->
-    {ok, #segment{} | undefined} | {error, term()}.
-scan_segment(Path, Id, MaxRecordBytes) ->
-    case file:open(Path, [read, binary]) of
-        {ok, Fd} ->
-            Result = scan_segment(Fd, Id, Path, MaxRecordBytes, 0, 0, 0),
-            _ = file:close(Fd),
-            Result;
-        {error, enoent} ->
-            {ok, undefined};
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec scan_segment(file:fd(), pos_integer(), file:filename_all(), pos_integer(),
-    non_neg_integer(), non_neg_integer(), non_neg_integer()) ->
-    {ok, #segment{} | undefined} | {error, term()}.
-scan_segment(Fd, Id, Path, MaxRecordBytes, StartSeq, EndSeq, Records) ->
-    case read_record(Fd, MaxRecordBytes) of
-        eof when Records =:= 0 ->
-            {ok, undefined};
-        eof ->
-            {ok, #segment{
-                id = Id,
-                path = Path,
-                start_seq = StartSeq,
-                end_seq = EndSeq,
-                records = Records
-            }};
-        {ok, Seq, _Packet} ->
-            NStartSeq = case StartSeq of
-                0 -> Seq;
-                _ -> StartSeq
-            end,
-            scan_segment(Fd, Id, Path, MaxRecordBytes, NStartSeq, Seq, Records + 1);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec encode_record(pos_integer(), binary()) -> binary().
-encode_record(Seq, Packet) ->
-    PacketSize = byte_size(Packet),
-    Magic = ?RECORD_MAGIC,
-    Header = <<Magic/binary, ?RECORD_VERSION:8/unsigned, ?RECORD_HEADER_SIZE:8/unsigned,
-        PacketSize:32/unsigned-big, Seq:64/unsigned-big>>,
-    Crc32 = erlang:crc32(<<Header/binary, Packet/binary>>),
-    <<Header/binary, Packet/binary, Crc32:32/unsigned-big>>.
-
--spec read_next_unacked(file:fd(), non_neg_integer(), pos_integer()) ->
-    eof | {ok, pos_integer(), binary()} | {error, term()}.
-read_next_unacked(Fd, AckedSeq, MaxRecordBytes) ->
-    case read_record(Fd, MaxRecordBytes) of
-        eof ->
-            eof;
-        {ok, Seq, Packet} when Seq > AckedSeq ->
-            {ok, Seq, Packet};
-        {ok, _Seq, _Packet} ->
-            read_next_unacked(Fd, AckedSeq, MaxRecordBytes);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec read_record(file:fd(), pos_integer()) -> eof | {ok, pos_integer(), binary()} | {error, term()}.
-read_record(Fd, MaxRecordBytes) ->
-    case find_next_magic(Fd) of
-        eof ->
-            eof;
-        {ok, StartPos} ->
-            case read_record_after_magic(Fd, MaxRecordBytes) of
-                {ok, Seq, Packet} ->
-                    {ok, Seq, Packet};
-                {resync, _Reason} ->
-                    case file:position(Fd, {bof, StartPos + 1}) of
-                        {ok, _} ->
-                            read_record(Fd, MaxRecordBytes);
-                        {error, Reason} ->
-                            {error, Reason}
-                    end
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec find_next_magic(file:fd()) -> eof | {ok, non_neg_integer()} | {error, term()}.
-find_next_magic(Fd) ->
-    case file:position(Fd, cur) of
-        {ok, StartPos} ->
-            case file:read(Fd, ?RECORD_MAGIC_SIZE) of
-                eof ->
-                    eof;
-                {ok, Magic} when byte_size(Magic) < ?RECORD_MAGIC_SIZE ->
-                    eof;
-                {ok, ?RECORD_MAGIC} ->
-                    {ok, StartPos};
-                {ok, Window} ->
-                    find_next_magic(Fd, Window);
-                {error, Reason} ->
-                    {error, Reason}
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec find_next_magic(file:fd(), binary()) -> eof | {ok, non_neg_integer()} | {error, term()}.
-find_next_magic(Fd, Window) ->
-    case file:read(Fd, 1) of
-        eof ->
-            eof;
-        {ok, Byte} ->
-            Tail = binary:part(Window, 1, ?RECORD_MAGIC_SIZE - 1),
-            NWindow = <<Tail/binary, Byte/binary>>,
-            case NWindow of
-                ?RECORD_MAGIC ->
-                    case file:position(Fd, cur) of
-                        {ok, Pos} ->
-                            {ok, Pos - ?RECORD_MAGIC_SIZE};
-                        {error, Reason} ->
-                            {error, Reason}
-                    end;
-                _ ->
-                    find_next_magic(Fd, NWindow)
-            end;
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec read_record_after_magic(file:fd(), pos_integer()) ->
-    {ok, pos_integer(), binary()} | {resync, term()}.
-read_record_after_magic(Fd, MaxRecordBytes) ->
-    HeaderRestSize = ?RECORD_HEADER_SIZE - ?RECORD_MAGIC_SIZE,
-    case file:read(Fd, HeaderRestSize) of
-        {ok, <<Version:8/unsigned, HeaderSize:8/unsigned, PacketSize:32/unsigned-big,
-            Seq:64/unsigned-big>>} ->
-            read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, MaxRecordBytes);
-        {ok, Partial} ->
-            {resync, {truncated_record_header, Partial}};
-        eof ->
-            {resync, truncated_record_header};
-        {error, Reason} ->
-            {resync, Reason}
-    end.
-
--spec read_record_payload(file:fd(), non_neg_integer(), non_neg_integer(),
-    non_neg_integer(), non_neg_integer(), pos_integer()) ->
-    {ok, pos_integer(), binary()} | {resync, term()}.
-read_record_payload(_Fd, Version, _HeaderSize, _PacketSize, _Seq, _MaxRecordBytes)
-  when Version =/= ?RECORD_VERSION ->
-    {resync, {invalid_record_version, Version}};
-read_record_payload(_Fd, _Version, HeaderSize, _PacketSize, _Seq, _MaxRecordBytes)
-  when HeaderSize =/= ?RECORD_HEADER_SIZE ->
-    {resync, {invalid_record_header_size, HeaderSize}};
-read_record_payload(_Fd, _Version, _HeaderSize, PacketSize, _Seq, MaxRecordBytes)
-  when PacketSize =:= 0; PacketSize > MaxRecordBytes ->
-    {resync, {invalid_record_packet_size, PacketSize}};
-read_record_payload(_Fd, _Version, _HeaderSize, _PacketSize, Seq, _MaxRecordBytes)
-  when Seq =:= 0 ->
-    {resync, {invalid_record_seq, Seq}};
-read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, _MaxRecordBytes) ->
-    case file:read(Fd, PacketSize + ?RECORD_CRC_SIZE) of
-        {ok, <<Packet:PacketSize/binary, Crc32:32/unsigned-big>>} ->
-            validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32);
-        {ok, Partial} ->
-            {resync, {truncated_record_payload, PacketSize, byte_size(Partial)}};
-        eof ->
-            {resync, {truncated_record_payload, PacketSize, 0}};
-        {error, Reason} ->
-            {resync, Reason}
-    end.
-
--spec validate_record_crc(non_neg_integer(), non_neg_integer(), non_neg_integer(),
-    pos_integer(), binary(), non_neg_integer()) ->
-    {ok, pos_integer(), binary()} | {resync, term()}.
-validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32) ->
-    Magic = ?RECORD_MAGIC,
-    Header = <<Magic/binary, Version:8/unsigned, HeaderSize:8/unsigned,
-        PacketSize:32/unsigned-big, Seq:64/unsigned-big>>,
-    case erlang:crc32(<<Header/binary, Packet/binary>>) of
-        Crc32 ->
-            {ok, Seq, Packet};
-        Expected ->
-            {resync, {crc32_mismatch, Seq, Expected, Crc32}}
-    end.

apps/efka/src/iot/efka_iot_stream.erl

@@ -1,156 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc One transparent TCP stream from iot to the local manager service.
-%%% The iot/efka protocol does not inspect HTTP; all HTTP bytes are carried
-%%% in data frames.
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_stream).
--include("message.hrl").
-
--export([start_stream/2]).
--export([run/2]).
-
--define(DEFAULT_CONNECT_TIMEOUT, 3000).
--define(DEFAULT_IDLE_TIMEOUT, 120000).
-
--type stream_id() :: pos_integer().
--type stream_target() :: pos_integer().
-
--spec start_stream(StreamId :: integer(), Target :: stream_target()) -> {ok, {pid(), reference()}}.
-start_stream(StreamId, Target) when is_integer(StreamId), StreamId > 0, is_integer(Target), Target > 0 ->
-    {ok, spawn_monitor(?MODULE, run, [StreamId, Target])}.
-
--spec run(StreamId :: stream_id(), Target :: stream_target()) -> ok.
-run(StreamId, Target) when is_integer(StreamId), StreamId > 0, is_integer(Target), Target > 0 ->
-    try run0(StreamId, Target) of
-        ok ->
-            ok
-    catch
-        Class:Reason:Stack ->
-            logger:warning("[efka_iot_stream] stream_id: ~p crashed, class: ~p, reason: ~p, stack: ~p",
-                [StreamId, Class, Reason, Stack]),
-            efka_iot_client:send_stream(StreamId, {reset, safe_term({Class, Reason})}),
-            ok
-    after
-        efka_iot_client:stream_done(StreamId)
-    end.
-
--spec run0(stream_id(), stream_target()) -> ok.
-run0(StreamId, Target) ->
-    case open_target_socket(Target) of
-        {ok, Socket, IdleTimeout} ->
-            efka_iot_client:send_stream(StreamId, opened),
-            ok = inet:setopts(Socket, [{active, once}]),
-            loop(StreamId, Socket, IdleTimeout);
-        {error, Reason} ->
-            efka_iot_client:send_stream(StreamId, {open_error, safe_term(Reason)}),
-            ok
-    end.
-
--spec open_target_socket(stream_target()) -> {ok, gen_tcp:socket(), timeout()} | {error, term()}.
-open_target_socket(Target) ->
-    case stream_target_props(Target) of
-        {ok, Props} ->
-            open_target_socket(Target, Props);
-        {error, Reason} ->
-            {error, Reason}
-    end.
-
--spec open_target_socket(stream_target(), proplists:proplist()) ->
-    {ok, gen_tcp:socket(), timeout()} | {error, term()}.
-open_target_socket(Target, Props) ->
-    Host = proplists:get_value(host, Props),
-    Port = proplists:get_value(port, Props),
-    ConnectTimeout = proplists:get_value(connect_timeout, Props, ?DEFAULT_CONNECT_TIMEOUT),
-    IdleTimeout = proplists:get_value(idle_timeout, Props, ?DEFAULT_IDLE_TIMEOUT),
-
-    SocketOpts = [
-        binary,
-        {packet, raw},
-        {active, false},
-        {nodelay, true}
-    ],
-    case gen_tcp:connect(Host, Port, SocketOpts, ConnectTimeout) of
-        {ok, Socket} ->
-            {ok, Socket, IdleTimeout};
-        {error, Reason} ->
-            {error, {connect_failed, Target, Reason}}
-    end.
-
--spec stream_target_props(stream_target()) -> {ok, proplists:proplist()} | {error, term()}.
-stream_target_props(Target) ->
-    case application:get_env(efka, stream_targets) of
-        {ok, Targets} ->
-            case proplists:get_value(Target, Targets) of
-                undefined ->
-                    {error, {unknown_stream_target, Target}};
-                Props ->
-                    {ok, Props}
-            end;
-        undefined when Target =:= ?STREAM_TARGET_MANAGER ->
-            case application:get_env(efka, stream_target) of
-                {ok, Props} ->
-                    {ok, Props};
-                undefined ->
-                    {error, {unknown_stream_target, Target}}
-            end;
-        undefined ->
-            {error, {unknown_stream_target, Target}}
-    end.
-
--spec loop(stream_id(), gen_tcp:socket(), timeout()) -> ok.
-loop(StreamId, Socket, IdleTimeout) ->
-    receive
-        {stream, StreamId, {data, Data}} when is_binary(Data) ->
-            case gen_tcp:send(Socket, Data) of
-                ok ->
-                    loop(StreamId, Socket, IdleTimeout);
-                {error, Reason} ->
-                    efka_iot_client:send_stream(StreamId, {reset, safe_term(Reason)}),
-                    close_socket(Socket)
-            end;
-        {stream, StreamId, fin} ->
-            _ = gen_tcp:shutdown(Socket, write),
-            loop(StreamId, Socket, IdleTimeout);
-        {stream, StreamId, {reset, _Reason}} ->
-            close_socket(Socket);
-        {tcp, Socket, Data} ->
-            efka_iot_client:send_stream(StreamId, {data, Data}),
-            ok = inet:setopts(Socket, [{active, once}]),
-            loop(StreamId, Socket, IdleTimeout);
-        {tcp_closed, Socket} ->
-            efka_iot_client:send_stream(StreamId, fin),
-            close_socket(Socket);
-        {tcp_error, Socket, Reason} ->
-            efka_iot_client:send_stream(StreamId, {reset, safe_term(Reason)}),
-            close_socket(Socket);
-        Info ->
-            logger:debug("[efka_iot_stream] stream_id: ~p ignore unknown info: ~p", [StreamId, Info]),
-            loop(StreamId, Socket, IdleTimeout)
-    after IdleTimeout ->
-        efka_iot_client:send_stream(StreamId, {reset, <<"idle_timeout">>}),
-        close_socket(Socket)
-    end.
-
--spec close_socket(gen_tcp:socket()) -> ok.
-close_socket(Socket) ->
-    catch gen_tcp:close(Socket),
-    ok.
-
--spec safe_term(term()) -> term().
-safe_term(true) ->
-    true;
-safe_term(false) ->
-    false;
-safe_term(undefined) ->
-    undefined;
-safe_term(Value) when is_atom(Value) ->
-    atom_to_binary(Value, utf8);
-safe_term(Value) when is_map(Value) ->
-    maps:from_list([{safe_term(K), safe_term(V)} || {K, V} <- maps:to_list(Value)]);
-safe_term(Value) when is_list(Value) ->
-    [safe_term(Item) || Item <- Value];
-safe_term(Value) when is_tuple(Value) ->
-    list_to_tuple([safe_term(Item) || Item <- tuple_to_list(Value)]);
-safe_term(Value) ->
-    Value.

apps/efka/src/message/message_codec.erl

@@ -0,0 +1,141 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 17. 9月 2025 16:05
+%%%-------------------------------------------------------------------
+-module(message_codec).
+-author("anlicheng").
+-include("message.hrl").
+
+-define(I8, 1).
+-define(I16, 2).
+-define(I32, 3).
+-define(Bytes, 4).
+
+%% API
+-export([encode/2, decode/1]).
+
+-spec encode(MessageType :: integer(), Message :: any()) -> binary().
+encode(MessageType, Message) when is_integer(MessageType) ->
+    Bin = encode0(Message),
+    <<MessageType, Bin/binary>>.
+encode0(#auth_request{uuid = UUID, username = Username, salt = Salt, token = Token, timestamp = Timestamp}) ->
+    iolist_to_binary([
+        marshal(?Bytes, UUID),
+        marshal(?Bytes, Username),
+        marshal(?Bytes, Salt),
+        marshal(?Bytes, Token),
+        marshal(?I32, Timestamp)
+    ]);
+encode0(#auth_reply{code = Code, payload = Payload}) ->
+    iolist_to_binary([
+        marshal(?I32, Code),
+        marshal(?Bytes, Payload)
+    ]);
+encode0(#jsonrpc_reply{result = Result, error = undefined}) ->
+    ResultBin = erlang:term_to_binary(#{<<"result">> => Result}),
+    iolist_to_binary([
+        marshal(?Bytes, ResultBin)
+    ]);
+encode0(#jsonrpc_reply{result = undefined, error = Error}) ->
+    ResultBin = erlang:term_to_binary(#{<<"error">> => Error}),
+    iolist_to_binary([
+        marshal(?Bytes, ResultBin)
+    ]);
+encode0(#pub{topic = Topic, qos = Qos, content = Content}) ->
+    iolist_to_binary([
+        marshal(?Bytes, Topic),
+        marshal(?I8, Qos),
+        marshal(?Bytes, Content)
+    ]);
+encode0(#command{command_type = CommandType, command = Command}) ->
+    iolist_to_binary([
+        marshal(?I32, CommandType),
+        marshal(?Bytes, Command)
+    ]);
+
+encode0(#jsonrpc_request{method = Method, params = Params}) ->
+    ReqBody = erlang:term_to_binary(#{<<"method">> => Method, <<"params">> => Params}),
+    iolist_to_binary([
+        marshal(?Bytes, ReqBody)
+    ]);
+encode0(#data{route_key = RouteKey, metric = Metric}) ->
+    iolist_to_binary([
+        marshal(?Bytes, RouteKey),
+        marshal(?Bytes, Metric)
+    ]);
+encode0(#task_event_stream{task_id = TaskId, type = Type, stream = Stream}) ->
+    iolist_to_binary([
+        marshal(?I32, TaskId),
+        marshal(?Bytes, Type),
+        marshal(?Bytes, Stream)
+    ]).
+
+-spec decode(Bin :: binary()) -> {ok, Message :: any()} | error.
+decode(<<PacketType:8, Packet/binary>>) ->
+    case unmarshal(Packet) of
+        {ok, Fields} ->
+            decode0(PacketType, Fields);
+        error ->
+            error
+    end.
+decode0(?MESSAGE_AUTH_REQUEST, [UUID, Username, Salt, Token, Timestamp]) ->
+    {ok, #auth_request{uuid = UUID, username = Username, salt = Salt, token = Token, timestamp = Timestamp}};
+decode0(?MESSAGE_JSONRPC_REPLY, [ReplyBin]) ->
+    case erlang:binary_to_term(ReplyBin) of
+        #{<<"result">> := Result} ->
+            {ok, #jsonrpc_reply{result = Result}};
+        #{<<"error">> := Error} ->
+            {ok, #jsonrpc_reply{error = Error}};
+        _ ->
+            error
+    end;
+decode0(?MESSAGE_PUB, [Topic, Qos, Content]) ->
+    {ok, #pub{topic = Topic, qos = Qos, content = Content}};
+decode0(?MESSAGE_COMMAND, [CommandType, Command]) ->
+    {ok, #command{command_type = CommandType, command = Command}};
+decode0(?MESSAGE_AUTH_REPLY, [Code, Payload]) ->
+    {ok, #auth_reply{code = Code, payload = Payload}};
+decode0(?MESSAGE_JSONRPC_REQUEST, [ReqBody]) ->
+    #{<<"method">> := Method, <<"params">> := Params} = erlang:binary_to_term(ReqBody),
+    {ok, #jsonrpc_request{method = Method, params = Params}};
+decode0(?MESSAGE_DATA, [RouteKey, Metric]) ->
+    {ok, #data{route_key = RouteKey, metric = Metric}};
+decode0(?MESSAGE_EVENT_STREAM, [TaskId, Type, Stream]) ->
+    {ok, #task_event_stream{task_id = TaskId, type = Type, stream = Stream}};
+decode0(_, _) ->
+    error.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%% helper methods
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+-spec marshal(Type :: integer(), Field :: any()) -> binary().
+marshal(?I8, Field) when is_integer(Field) ->
+    <<?I8, Field:8>>;
+marshal(?I16, Field) when is_integer(Field) ->
+    <<?I16, Field:16>>;
+marshal(?I32, Field) when is_integer(Field) ->
+    <<?I32, Field:32>>;
+marshal(?Bytes, Field) when is_binary(Field) ->
+    Len = byte_size(Field),
+    <<?Bytes, Len:16, Field/binary>>.
+
+-spec unmarshal(Bin :: binary()) -> {ok, Components :: [any()]} | error.
+unmarshal(Bin) when is_binary(Bin) ->
+    unmarshal(Bin, []).
+unmarshal(<<>>, Acc) ->
+    {ok, lists:reverse(Acc)};
+unmarshal(<<?I8, F:8, Rest/binary>>, Acc) ->
+    unmarshal(Rest, [F|Acc]);
+unmarshal(<<?I16, F:16, Rest/binary>>, Acc) ->
+    unmarshal(Rest, [F|Acc]);
+unmarshal(<<?I32, F:32, Rest/binary>>, Acc) ->
+    unmarshal(Rest, [F|Acc]);
+unmarshal(<<?Bytes, Len:16, F:Len/binary, Rest/binary>>, Acc) ->
+    unmarshal(Rest, [F|Acc]);
+unmarshal(_, _) ->
+    error.

apps/efka/src/models/cache_model.erl

@@ -0,0 +1,142 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 13. 8月 2025 16:05
+%%%-------------------------------------------------------------------
+-module(cache_model).
+-author("anlicheng").
+
+-behaviour(gen_server).
+
+%% API
+-export([start_link/0]).
+-export([insert/1, fetch_next/0, delete/1, get_all_cache/0]).
+
+%% gen_server callbacks
+-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
+
+-define(SERVER, ?MODULE).
+-define(TAB, cache).
+
+-record(state, {
+
+}).
+
+%%%===================================================================
+%%% API
+%%%===================================================================
+
+-spec insert(Data :: binary()) -> ok | {error, Reason :: any()}.
+insert(Data) when is_binary(Data) ->
+    gen_server:call(?SERVER, {insert, {generate_id(), Data}}).
+
+fetch_next() ->
+    gen_server:call(?SERVER, fetch_next).
+
+delete(Id) when is_integer(Id) ->
+    gen_server:call(?SERVER, {delete, Id}).
+
+-spec get_all_cache() -> [binary()].
+get_all_cache() ->
+    gen_server:call(?SERVER, get_all_cache).
+
+%% @doc Spawns the server and registers the local name (unique)
+-spec(start_link() ->
+    {ok, Pid :: pid()} | ignore | {error, Reason :: term()}).
+start_link() ->
+    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
+
+%%%===================================================================
+%%% gen_server callbacks
+%%%===================================================================
+
+%% @private
+%% @doc Initializes the server
+-spec(init(Args :: term()) ->
+    {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term()} | ignore).
+init([]) ->
+    {ok, DetsDir} = application:get_env(efka, dets_dir),
+    File = DetsDir ++ "cache.dets",
+    {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, bag}, {keypos, 1}]),
+    {ok, #state{}}.
+
+%% @private
+%% @doc Handling call messages
+-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()},
+    State :: #state{}) ->
+    {reply, Reply :: term(), NewState :: #state{}} |
+    {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} |
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_call({insert, Cache}, _From, State = #state{}) ->
+    ok = dets:insert(?TAB, Cache),
+    {reply, ok, State};
+handle_call(fetch_next, _From, State = #state{}) ->
+    case dets:first(?TAB) of
+        '$end_of_table' ->
+            {reply, error, State};
+        Key ->
+            [Entry] = dets:lookup(?TAB, Key),
+            {reply, {ok, Entry}, State}
+    end;
+handle_call({delete, Id}, _From, State = #state{}) ->
+    ok = dets:delete(?TAB, Id),
+    {reply, ok, State};
+
+handle_call(get_all_cache, _From, State = #state{}) ->
+    Items = dets:foldl(fun(Record, Acc) -> [Record|Acc] end, [], ?TAB),
+    {reply, {ok, lists:reverse(Items)}, State};
+
+handle_call(_Request, _From, State = #state{}) ->
+    {reply, ok, State}.
+
+%% @private
+%% @doc Handling cast messages
+-spec(handle_cast(Request :: term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_cast(_Request, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc Handling all non call/cast messages
+-spec(handle_info(Info :: timeout() | term(), State :: #state{}) ->
+    {noreply, NewState :: #state{}} |
+    {noreply, NewState :: #state{}, timeout() | hibernate} |
+    {stop, Reason :: term(), NewState :: #state{}}).
+handle_info(_Info, State = #state{}) ->
+    {noreply, State}.
+
+%% @private
+%% @doc This function is called by a gen_server when it is about to
+%% terminate. It should be the opposite of Module:init/1 and do any
+%% necessary cleaning up. When it returns, the gen_server terminates
+%% with Reason. The return value is ignored.
+-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()),
+    State :: #state{}) -> term()).
+terminate(_Reason, _State = #state{}) ->
+    dets:close(?TAB),
+    ok.
+
+%% @private
+%% @doc Convert process state when code is changed
+-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{},
+    Extra :: term()) ->
+    {ok, NewState :: #state{}} | {error, Reason :: term()}).
+code_change(_OldVsn, State = #state{}, _Extra) ->
+    {ok, State}.
+
+%%%===================================================================
+%%% Internal functions
+%%%===================================================================
+
+-spec generate_id() -> integer().
+generate_id() ->
+    os:system_time(microsecond).

apps/efka/src/models/service_model.erl

@@ -6,7 +6,7 @@
 %%% @end
 %%% Created : 13. 8月 2025 16:41
 %%%-------------------------------------------------------------------
--module(efka_service_model).
+-module(service_model).
 -author("anlicheng").
 -include("efka_tables.hrl").
 
@@ -30,11 +30,9 @@
 %%% API
 %%%===================================================================
 
--spec insert(#service{}) -> ok.
 insert(Service = #service{}) ->
     gen_server:call(?SERVER, {insert, Service}).
 
--spec change_status(binary(), integer()) -> ok | {error, binary()}.
 change_status(ServiceId, NewStatus) when is_binary(ServiceId), is_integer(NewStatus) ->
     gen_server:call(?SERVER, {change_status, ServiceId, NewStatus}).
 

apps/efka/src/service/efka_service_channel.erl

@@ -1,169 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @author licheng5
-%%% @copyright (C) 2021, <COMPANY>
-%%% @doc
-%%%
-%%% @end
-%%% Created : 11. 1月 2021 上午12:17
-%%%-------------------------------------------------------------------
--module(efka_service_channel).
--author("licheng5").
--include("efka_tables.hrl").
--include("service_pb.hrl").
-
-%% 一级帧类型
-%% REQUEST: 需要响应的请求帧
-%% RESPONSE: 对 REQUEST 的响应帧
-%% CAST: 单向消息帧，不需要响应
--define(FRAME_REQUEST, 16#01).
--define(FRAME_REPLY, 16#02).
--define(FRAME_CAST, 16#03).
-
-%% API
--export([init/2]).
--export([websocket_init/1, websocket_handle/2, websocket_info/2, terminate/3]).
-
--record(state, {
-    service_id :: undefined | binary(),
-    service_pid :: undefined | pid(),
-    subscribed_topics = sets:new(),
-    is_registered = false :: boolean()
-}).
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%% 逻辑处理方法
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
--spec init(term(), term()) -> {cowboy_websocket, term(), term()}.
-init(Req, Opts) ->
-    {cowboy_websocket, Req, Opts}.
-
--spec websocket_init(term()) -> {ok, #state{}}.
-websocket_init(_State) ->
-    logger:debug("[efka_service_channel] get a new connection"),
-    %% 初始状态为true
-    {ok, #state{}}.
-
--spec websocket_handle(term(), #state{}) ->
-    {reply, term(), #state{}} | {ok, #state{}}.
-websocket_handle(ping, State) ->
-    {reply, pong, State};
-
-websocket_handle({binary, <<?FRAME_REQUEST, PacketBin/binary>>}, State) ->
-    Request = service_pb:decode_msg(PacketBin, 'ServiceRequest'),
-    logger:debug("[efka_service_channel] get request: ~p", [Request]),
-    handle_request(Request, State);
-websocket_handle({binary, <<?FRAME_CAST, PacketBin/binary>>}, State) ->
-    Cast = service_pb:decode_msg(PacketBin, 'ServiceCast'),
-    logger:debug("[efka_service_channel] get cast: ~p", [Cast]),
-    handle_cast(Cast, State);
-
-websocket_handle(Info, State) ->
-    logger:error("[efka_service_channel] get a unknown message: ~p, channel will closed", [Info]),
-    {ok, State}.
-
-%% 订阅的消息
--spec websocket_info(term(), #state{}) ->
-    {reply, term(), #state{}} | {stop, #state{}} | {ok, #state{}}.
-websocket_info({topic_broadcast, Topic, Content}, State = #state{}) ->
-    Packet = service_pb:encode_msg(#'ServiceCast'{
-        body = {topic_event, #'ServiceCast.TopicEvent'{topic = Topic, content = Content}}
-    }),
-    logger:debug("[efka_service_channel] will publish topic: ~p", [Topic]),
-    {reply, {binary, <<?FRAME_CAST, Packet/binary>>}, State};
-
-%% service进程关闭
-websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) ->
-    logger:debug("[efka_service_channel] container_pid: ~p, exited: ~p", [ServicePid, Reason]),
-    {stop, State#state{service_pid = undefined}};
-
-%% 处理关闭信号
-websocket_info({stop, Reason}, State) ->
-    logger:debug("[efka_service_channel] the channel will be closed with reason: ~p", [Reason]),
-    {stop, State};
-
-%% 处理其他未知消息
-websocket_info(Info, State) ->
-    logger:debug("[efka_service_channel] channel get unknown info: ~p", [Info]),
-    {ok, State}.
-
-%% 进程关闭事件
--spec terminate(term(), term(), #state{}) -> ok.
-terminate(Reason, _Req, State = #state{service_id = ServiceId, is_registered = IsRegistered}) ->
-    ok = efka_subscription:unsubscribe_all(self()),
-    case IsRegistered of
-        true ->
-            ok = efka_service_model:change_status(ServiceId, 0);
-        false ->
-            ok
-    end,
-    logger:debug("[efka_service_channel] channel close with reason: ~p, state is: ~p", [Reason, State]),
-    ok.
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%% helper methods
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-%% 注册, 要建立程序和容器之间的关系
--spec handle_request(service_pb:'ServiceRequest'(), #state{}) -> {reply, {binary, binary()}, #state{}}.
-handle_request(#'ServiceRequest'{packet_id = PacketId, request = {register, #'ServiceRequest.Register'{service_id = ServiceId}}}, State) ->
-    {ok, ServicePid} = efka_service_sup:start_service(ServiceId),
-    case efka_service:attach_channel(ServicePid, self()) of
-        ok ->
-            erlang:monitor(process, ServicePid),
-
-            %% 更新微服务的状态
-            ok = efka_service_model:insert(#service{
-                service_id = ServiceId,
-                container_name = <<>>,
-                status = ?SERVICE_RUNNING,
-                meta_data = #{},
-                create_ts = efka_util:timestamp(),
-                update_ts = efka_util:timestamp()
-            }),
-
-            {reply, {binary, result_reply_packet(PacketId, <<"ok">>)},
-                State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}};
-        {error, Error} ->
-            logger:warning("[efka_service_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]),
-            {reply, {binary, error_reply_packet(PacketId, -1, <<"attach channel failed">>)}, State}
-    end;
-
-%% 订阅事件
-handle_request(#'ServiceRequest'{packet_id = PacketId, request = {subscribe, #'ServiceRequest.Subscribe'{topic = Topic}}},
-    State = #state{subscribed_topics = SubscribedTopics, is_registered = true}) ->
-
-    case efka_subscription:subscribe(Topic, self()) of
-        ok ->
-            Packet = result_reply_packet(PacketId, <<"ok">>),
-            {reply, {binary, Packet}, State#state{subscribed_topics = sets:add_element(Topic, SubscribedTopics)}};
-        {error, Reason} ->
-            Packet = error_reply_packet(PacketId, -1, Reason),
-            {reply, {binary, Packet}, State}
-    end;
-handle_request(#'ServiceRequest'{packet_id = PacketId}, State) ->
-    {reply, {binary, error_reply_packet(PacketId, -1, <<"invalid request">>)}, State}.
-
--spec handle_cast(service_pb:'ServiceCast'(), #state{}) -> {ok, #state{}}.
-handle_cast(#'ServiceCast'{body = {metric_data, #'ServiceCast.MetricData'{route_key = RouteKey, metric = Metric}}},
-    State = #state{service_pid = ServicePid, is_registered = true}) ->
-    efka_service:metric_data(ServicePid, RouteKey, Metric),
-    {ok, State};
-handle_cast(#'ServiceCast'{body = _Body}, State) ->
-    {ok, State}.
-
--spec result_reply_packet(integer(), binary()) -> binary().
-result_reply_packet(PacketId, Result) when is_integer(PacketId), is_binary(Result) ->
-    Reply = service_pb:encode_msg(#'ServiceReply'{
-        packet_id = PacketId,
-        reply = {result, Result}
-    }),
-    <<?FRAME_REPLY, Reply/binary>>.
-
--spec error_reply_packet(integer(), integer(), binary()) -> binary().
-error_reply_packet(PacketId, Code, Message) when is_integer(PacketId), is_integer(Code), is_binary(Message) ->
-    Reply = service_pb:encode_msg(#'ServiceReply'{
-        packet_id = PacketId,
-        reply = {error, #'ServiceReply.Error'{code = Code, message = Message}}
-    }),
-    <<?FRAME_REPLY, Reply/binary>>.

apps/efka/src/tests/docker_commands_tests.erl

@@ -0,0 +1,104 @@
+%%%-------------------------------------------------------------------
+%%% @author anlicheng
+%%% @copyright (C) 2025, <COMPANY>
+%%% @doc
+%%%
+%%% @end
+%%% Created : 23. 9月 2025 17:23
+%%%-------------------------------------------------------------------
+-module(docker_commands_tests).
+-author("anlicheng").
+
+%% API
+-export([test_pull/0, test_commands/0, test_create_container/0]).
+
+test_pull() ->
+    Image = <<"docker.1ms.run/library/nginx:latest">>,
+    docker_commands:pull_image(Image, fun(Msg) -> lager:debug("msg is: ~p", [Msg]) end).
+
+test_commands() ->
+    Id = <<"redpanda-console">>,
+    StopRes = docker_commands:stop_container(Id),
+    lager:debug("stop res: ~p", [StopRes]),
+    StartRes = docker_commands:start_container(Id),
+    lager:debug("start res: ~p", [StartRes]).
+
+test_create_container() ->
+    M = #{
+        <<"image">> => <<"docker.1ms.run/library/nginx:latest">>,
+        <<"container_name">> => <<"my_nginx_new1">>,
+        <<"command">> => [
+            <<"nginx">>,
+            <<"-g">>,
+            <<"daemon off;">>
+        ],
+        <<"entrypoint">> => [
+            <<"/docker-entrypoint.sh">>
+        ],
+        <<"envs">> => [
+            <<"ENV1=val1">>,
+            <<"ENV2=val2">>
+        ],
+        <<"env_file">> => [
+            <<"./env.list">>
+        ],
+        <<"ports">> => [
+            <<"8080:80">>,
+            <<"443:443">>
+        ],
+        <<"expose">> => [
+            <<"80">>,
+            <<"443">>
+        ],
+        <<"volumes">> => [
+            <<"/host/data:/data">>,
+            <<"/host/log:/var/log">>
+        ],
+        <<"networks">> => [
+            <<"mynet">>
+        ],
+        <<"labels">> => #{
+            <<"role">> => <<"web">>,
+            <<"env">> => <<"prod">>
+        },
+        <<"restart">> => <<"always">>,
+        <<"user">> => <<"www-data">>,
+        <<"working_dir">> => <<"/app">>,
+        <<"hostname">> => <<"myhost">>,
+        <<"privileged">> => true,
+        <<"cap_add">> => [
+            <<"NET_ADMIN">>
+        ],
+        <<"cap_drop">> => [
+            <<"MKNOD">>
+        ],
+        <<"devices">> => [
+            <<"/dev/snd:/dev/snd">>
+        ],
+        <<"mem_limit">> => <<"512m">>,
+        <<"mem_reservation">> => <<"256m">>,
+        <<"cpu_shares">> => 512,
+        <<"cpus">> => 1.5,
+        <<"ulimits">> => #{
+            <<"nofile">> => <<"1024:2048">>
+        },
+        <<"sysctls">> => #{
+            <<"net.ipv4.ip_forward">> => <<"1">>
+        },
+        <<"tmpfs">> => [
+            <<"/tmp">>
+        ],
+        <<"extra_hosts">> => [
+            <<"host1:192.168.0.1">>
+        ],
+        <<"healthcheck">> => #{
+            <<"test">> => [
+                <<"CMD-SHELL">>,
+                <<"curl -f http://localhost || exit 1">>
+            ],
+            <<"interval">> => <<"30s">>,
+            <<"timeout">> => <<"10s">>,
+            <<"retries">> => 3
+        }
+    },
+    docker_commands:create_container(<<"my_nginx_xx3">>, "/usr/local/code/efka/", M).

apps/efka/src/tests/efka_iot_outbox_test.erl

@@ -1,315 +0,0 @@
-%%%-------------------------------------------------------------------
-%%% @doc
-%%% efka_iot_outbox 的手动模拟测试模块。
-%%%
-%%% 该模块基于 gen_server 实现，用两个定时 tick 模拟一边写入、一边读取：
-%%% - write_tick 每次向 efka_iot_outbox 写入一条 packet。
-%%% - read_tick 每次从 efka_iot_outbox 读取并 ack 一条 packet。
-%%% - 默认 read_interval = write_interval * 2，因此读取速度是写入速度的
-%%%   50%。
-%%% - 读取到的数据会通过 logger:debug/2 输出。
-%%%
-%%% 默认使用独立临时 dir，避免污染真实运行目录。也可以通过 start_link/1
-%%% 传入 {dir, Dir} 指定 outbox 目录。
-%%%
-%%% 示例：
-%%% <pre>
-%%% efka_iot_outbox_test:start_link([
-%%%     {dir, "/private/tmp/efka_iot_outbox_test"},
-%%%     {write_interval, 100},
-%%%     {read_interval, 200},
-%%%     {max_writes, 1000}
-%%% ]).
-%%% efka_iot_outbox_test:stats().
-%%% efka_iot_outbox_test:stop().
-%%% </pre>
-%%% @end
-%%%-------------------------------------------------------------------
--module(efka_iot_outbox_test).
-
--behaviour(gen_server).
-
--export([start/0, start/1, start_link/0, start_link/1, stop/0, stats/0]).
--export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
--export([test/0]).
-
--define(SERVER, ?MODULE).
--define(DEFAULT_WRITE_INTERVAL, 100).
--define(DEFAULT_SEGMENT_RECORD_LIMIT, 2000).
--define(DEFAULT_MAX_SEGMENTS, 5).
--define(DEFAULT_MAX_RECORD_BYTES, 16 * 1024 * 1024).
-
--record(state, {
-    outbox :: efka_iot_outbox:outbox(),
-    dir :: file:filename_all(),
-    write_interval = ?DEFAULT_WRITE_INTERVAL :: pos_integer(),
-    read_interval = ?DEFAULT_WRITE_INTERVAL * 2 :: pos_integer(),
-    max_writes = infinity :: pos_integer() | infinity,
-    write_timer = undefined :: undefined | reference(),
-    read_timer = undefined :: undefined | reference(),
-    write_count = 0 :: non_neg_integer(),
-    appended_count = 0 :: non_neg_integer(),
-    read_count = 0 :: non_neg_integer(),
-    dropped_count = 0 :: non_neg_integer(),
-    error_count = 0 :: non_neg_integer()
-}).
-
--type option() ::
-    {dir, file:filename_all()} |
-    {write_interval, pos_integer()} |
-    {read_interval, pos_integer()} |
-    {max_writes, pos_integer() | infinity} |
-    {segment_record_limit, pos_integer()} |
-    {max_segments, pos_integer()} |
-    {max_record_bytes, pos_integer()}.
-
-
-test() ->
-    efka_iot_outbox_test:start_link([
-        {write_interval, 10},
-        {read_interval, 20},
-        {max_writes, 100_0000}
-    ]).
-
--spec start() -> {ok, pid()} | {error, term()}.
-start() ->
-    start([]).
-
--spec start([option()] | map()) -> {ok, pid()} | {error, term()}.
-start(Options) ->
-    gen_server:start({local, ?SERVER}, ?MODULE, Options, []).
-
--spec start_link() -> {ok, pid()} | {error, term()}.
-start_link() ->
-    start_link([]).
-
--spec start_link([option()] | map()) -> {ok, pid()} | {error, term()}.
-start_link(Options) ->
-    gen_server:start_link({local, ?SERVER}, ?MODULE, Options, []).
-
--spec stop() -> ok.
-stop() ->
-    gen_server:stop(?SERVER).
-
--spec stats() -> map().
-stats() ->
-    gen_server:call(?SERVER, stats).
-
--spec init([option()] | map()) -> {ok, #state{}} | {stop, term()}.
-init(Options0) ->
-    process_flag(trap_exit, true),
-    Options = normalize_options(Options0),
-    WriteInterval = positive_option(write_interval, Options, ?DEFAULT_WRITE_INTERVAL),
-    ReadInterval = positive_option(read_interval, Options, WriteInterval * 2),
-    SegmentRecordLimit = positive_option(segment_record_limit, Options, ?DEFAULT_SEGMENT_RECORD_LIMIT),
-    MaxSegments = positive_option(max_segments, Options, ?DEFAULT_MAX_SEGMENTS),
-    MaxRecordBytes = positive_option(max_record_bytes, Options, ?DEFAULT_MAX_RECORD_BYTES),
-    MaxWrites = max_writes_option(Options),
-
-    Dir = "/private/tmp/efka_outbox/",
-    ok = filelib:ensure_dir(filename:join(Dir, "dummy")),
-    OutboxOptions = #{
-        dir => Dir,
-        segment_record_limit => SegmentRecordLimit,
-        max_segments => MaxSegments,
-        max_record_bytes => MaxRecordBytes
-    },
-    case efka_iot_outbox:open(OutboxOptions) of
-        {ok, Outbox} ->
-            State0 = #state{
-                outbox = Outbox,
-                dir = Dir,
-                write_interval = WriteInterval,
-                read_interval = ReadInterval,
-                max_writes = MaxWrites
-            },
-            State1 = schedule_write(State0, 0),
-            State2 = schedule_read(State1, ReadInterval),
-            {ok, State2};
-        {error, Reason} ->
-            {stop, Reason}
-    end.
-
--spec handle_call(term(), gen_server:from(), #state{}) ->
-    {reply, term(), #state{}}.
-handle_call(stats, _From, State) ->
-    {reply, state_stats(State), State};
-handle_call(_Request, _From, State) ->
-    {reply, {error, unknown_call}, State}.
-
--spec handle_cast(term(), #state{}) -> {noreply, #state{}}.
-handle_cast(_Request, State) ->
-    {noreply, State}.
-
--spec handle_info(term(), #state{}) -> {noreply, #state{}}.
-handle_info(write_tick, State0 = #state{write_timer = WriteTimer}) ->
-    State1 = State0#state{write_timer = undefined},
-    State2 = case can_write(State1) of
-        true ->
-            append_one(State1);
-        false ->
-            State1
-    end,
-    State3 = case can_write(State2) of
-        true ->
-            schedule_write(State2, State2#state.write_interval);
-        false ->
-            State2
-    end,
-    _ = WriteTimer,
-    {noreply, State3};
-handle_info(read_tick, State0 = #state{read_timer = ReadTimer}) ->
-    State1 = State0#state{read_timer = undefined},
-    State2 = read_one(State1),
-    State3 = schedule_read(State2, State2#state.read_interval),
-    _ = ReadTimer,
-    {noreply, State3};
-handle_info(_Info, State) ->
-    {noreply, State}.
-
--spec terminate(term(), #state{}) -> ok.
-terminate(_Reason, #state{outbox = Outbox, write_timer = WriteTimer, read_timer = ReadTimer}) ->
-    cancel_timer(WriteTimer),
-    cancel_timer(ReadTimer),
-    efka_iot_outbox:close(Outbox),
-    ok.
-
--spec code_change(term(), #state{}, term()) -> {ok, #state{}}.
-code_change(_OldVsn, State, _Extra) ->
-    {ok, State}.
-
--spec append_one(#state{}) -> #state{}.
-append_one(State = #state{outbox = Outbox, write_count = WriteCount}) ->
-    PacketId = WriteCount + 1,
-    Packet = term_to_binary(#{
-        source => efka_iot_outbox_test,
-        packet_id => PacketId,
-        monotonic_time => erlang:monotonic_time(millisecond)
-    }),
-    case efka_iot_outbox:append(Packet, Outbox) of
-        {ok, NOutbox} ->
-            State#state{
-                outbox = NOutbox,
-                write_count = PacketId,
-                appended_count = State#state.appended_count + 1
-            };
-        {dropped, capacity_reached, NOutbox} ->
-            logger:warning("[efka_iot_outbox_test] outbox capacity reached, packet_id: ~p", [PacketId]),
-            State#state{
-                outbox = NOutbox,
-                write_count = PacketId,
-                dropped_count = State#state.dropped_count + 1
-            };
-        {error, Reason} ->
-            logger:warning("[efka_iot_outbox_test] append failed, packet_id: ~p, reason: ~p", [PacketId, Reason]),
-            State#state{
-                write_count = PacketId,
-                error_count = State#state.error_count + 1
-            }
-    end.
-
--spec read_one(#state{}) -> #state{}.
-read_one(State = #state{outbox = Outbox}) ->
-    case efka_iot_outbox:next(Outbox) of
-        {ok, Seq, Packet} ->
-            logger:debug("[efka_iot_outbox_test] read seq: ~p, packet: ~p", [Seq, decode_packet(Packet)]),
-            case efka_iot_outbox:ack(Seq, Outbox) of
-                {ok, NOutbox} ->
-                    State#state{
-                        outbox = NOutbox,
-                        read_count = State#state.read_count + 1
-                    };
-                {error, Reason} ->
-                    logger:warning("[efka_iot_outbox_test] ack failed, seq: ~p, reason: ~p", [Seq, Reason]),
-                    State#state{error_count = State#state.error_count + 1}
-            end;
-        eof ->
-            State;
-        {error, Reason} ->
-            logger:warning("[efka_iot_outbox_test] read failed, reason: ~p", [Reason]),
-            State#state{error_count = State#state.error_count + 1}
-    end.
-
--spec schedule_write(#state{}, non_neg_integer()) -> #state{}.
-schedule_write(State, Delay) ->
-    Ref = erlang:send_after(Delay, self(), write_tick),
-    State#state{write_timer = Ref}.
-
--spec schedule_read(#state{}, non_neg_integer()) -> #state{}.
-schedule_read(State, Delay) ->
-    Ref = erlang:send_after(Delay, self(), read_tick),
-    State#state{read_timer = Ref}.
-
--spec cancel_timer(undefined | reference()) -> ok.
-cancel_timer(undefined) ->
-    ok;
-cancel_timer(Ref) ->
-    _ = erlang:cancel_timer(Ref),
-    ok.
-
--spec can_write(#state{}) -> boolean().
-can_write(#state{max_writes = infinity}) ->
-    true;
-can_write(#state{write_count = WriteCount, max_writes = MaxWrites}) ->
-    WriteCount < MaxWrites.
-
--spec state_stats(#state{}) -> map().
-state_stats(#state{
-    dir = Dir,
-    write_interval = WriteInterval,
-    read_interval = ReadInterval,
-    max_writes = MaxWrites,
-    write_count = WriteCount,
-    appended_count = AppendedCount,
-    read_count = ReadCount,
-    dropped_count = DroppedCount,
-    error_count = ErrorCount
-}) ->
-    #{
-        dir => Dir,
-        write_interval => WriteInterval,
-        read_interval => ReadInterval,
-        max_writes => MaxWrites,
-        write_count => WriteCount,
-        appended_count => AppendedCount,
-        read_count => ReadCount,
-        dropped_count => DroppedCount,
-        error_count => ErrorCount,
-        pending_count => AppendedCount - ReadCount
-    }.
-
--spec normalize_options([option()] | map()) -> map().
-normalize_options(Options) when is_map(Options) ->
-    Options;
-normalize_options(Options) when is_list(Options) ->
-    maps:from_list(Options).
-
--spec positive_option(atom(), map(), pos_integer()) -> pos_integer().
-positive_option(Key, Options, Default) ->
-    case maps:get(Key, Options, Default) of
-        Value when is_integer(Value), Value > 0 ->
-            Value;
-        Value ->
-            error({invalid_option, Key, Value})
-    end.
-
--spec max_writes_option(map()) -> pos_integer() | infinity.
-max_writes_option(Options) ->
-    case maps:get(max_writes, Options, infinity) of
-        infinity ->
-            infinity;
-        Value when is_integer(Value), Value > 0 ->
-            Value;
-        Value ->
-            error({invalid_option, max_writes, Value})
-    end.
-
--spec decode_packet(binary()) -> term().
-decode_packet(Packet) ->
-    try binary_to_term(Packet, [safe]) of
-        Term ->
-            Term
-    catch
-        error:_ ->
-            Packet
-    end.

config/sys.config

@@ -0,0 +1,69 @@
+[
+  {efka, [
+    {root_dir, "/usr/local/code/efka"},
+
+    {dets_dir, "/usr/local/code/tmp/dets/"},
+
+    {upload_dir, "/usr/local/code/tmp/upload/"},
+
+    {tcp_server, [
+      {port, 18088}
+    ]},
+
+    {http_server, [
+      {port, 18080},
+      {acceptors, 10},
+      {max_connections, 1024},
+      {backlog, 256}
+    ]},
+
+    {tls_server, [
+      {host, "localhost"},
+      {port, 443}
+    ]},
+
+    {auth, [
+      {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"},
+      {username, "test"},
+      {salt, "salt2345"},
+      {token, "token124"}
+    ]}
+
+  ]},
+
+  %% 系统日志配置，系统日志为lager, 支持日志按日期自动分割
+  {lager, [
+    {colored, true},
+    %% Whether to write a crash log, and where. Undefined means no crash logger.
+    {crash_log, "trade_hub.crash.log"},
+    %% Maximum size in bytes of events in the crash log - defaults to 65536
+    {crash_log_msg_size, 65536},
+    %% Maximum size of the crash log in bytes, before its rotated, set
+    %% to 0 to disable rotation - default is 0
+    {crash_log_size, 10485760},
+    %% What time to rotate the crash log - default is no time
+    %% rotation. See the README for a description of this format.
+    {crash_log_date, "$D0"},
+    %% Number of rotated crash logs to keep, 0 means keep only the
+    %% current one - default is 0
+    {crash_log_count, 5},
+    %% Whether to redirect error_logger messages into lager - defaults to true
+    {error_logger_redirect, true},
+
+    %% How big the gen_event mailbox can get before it is switched into sync mode
+    {async_threshold, 20},
+    %% Switch back to async mode, when gen_event mailbox size decrease from `async_threshold'
+    %% to async_threshold - async_threshold_window
+    {async_threshold_window, 5},
+
+    {handlers, [
+      %% debug | info | warning | error, 日志级别
+      {lager_console_backend, debug},
+      {lager_file_backend, [{file, "debug.log"}, {level, debug}, {size, 314572800}]},
+      {lager_file_backend, [{file, "notice.log"}, {level, notice}, {size, 314572800}]},
+      {lager_file_backend, [{file, "error.log"}, {level, error}, {size, 314572800}]},
+      {lager_file_backend, [{file, "info.log"}, {level, info}, {size, 314572800}]}
+    ]}
+
+  ]}
+].

config/sys.config.src

@@ -1,70 +0,0 @@
-[
-  {efka, [
-    {dets_dir, "${EFKA_DETS_DIR}"},
-
-    {websocket_server, [
-      {port, 18080},
-      {acceptors, 10},
-      {max_connections, 1024},
-      {backlog, 256}
-    ]},
-
-    {iot_server, [
-      {host, "${EFKA_IOT_HOST}"},
-      {tls_port, 1443},
-      {udp_port, 24000}
-    ]},
-
-    {stream_targets, [
-      {1, [
-        {name, manager},
-        {host, "127.0.0.1"},
-        {port, 81},
-        {connect_timeout, 3000},
-        {idle_timeout, 120000}
-      ]}
-    ]},
-
-    {heartbeat, [
-      {interval, 5000}
-    ]},
-
-    {auth, [
-      {uuid, "${EFKA_AUTH_UUID}"},
-      {token, "${EFKA_AUTH_TOKEN}"}
-    ]}
-
-  ]},
-
-  {docker, [
-    {root_dir, "${EFKA_DOCKER_ROOT_DIR}"}
-  ]},
-
-  %% 系统日志配置，使用 OTP logger
-  {kernel, [
-    %% 设置 Logger 的 primary log level
-    {logger_level, debug},
-    {logger, [
-      {handler, default, logger_std_h,
-        #{
-          level => debug,
-          formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}}
-        }
-      },
-
-      {handler, disk, logger_disk_log_h,
-        #{
-          level => debug,
-          config => #{
-            file => "log/debug.log",
-            max_no_files => 10,
-            max_no_bytes => 524288000
-          },
-          formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}}
-        }
-      }
-
-    ]}
-  ]}
-
-].

config/vm.args

@@ -1,9 +1,9 @@
--sname efka
+-name efka
 
 -setcookie efka_cookie
 -kernel start_group false
 
--mnesia dir '"${EFKA_MNESIA_DIR}"'
+-mnesia dir '"/usr/local/var/mnesia/efka"'
 -mnesia dump_log_write_threshold 5000
 -mnesia dc_dump_limit 40
 

docs/container_command_docker_json.md

@@ -1,803 +0,0 @@
-# IOT 容器命令到 Docker JSON 的转换说明
-
-本文档描述 `efka` 收到 `iot` 下发的容器管理 command 后，接受的 Erlang map 格式，以及 deploy 命令中 `create` 参数如何转换成 Docker Engine API 接收的 JSON。
-
-对应代码：
-
-- command 接收入口：[src/iot/efka_iot_client.erl](/usr/local/code/cloudkit/efka/apps/efka/src/iot/efka_iot_client.erl:177)
-- 部署任务管理：[docker_deploy_manager.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deploy_manager.erl:36)
-- 部署执行：[docker_deployer.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deployer.erl:39)
-- Docker JSON 构造：[docker_container_builder.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_container_builder.erl:14)
-- Docker API 调用：[docker_commands.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_commands.erl:36)
-
-## 1. 协议入口
-
-`iot` 通过 TLS 长连接向 `efka` 下发容器命令：
-
-```erlang
-{<<"command">>, Ref, {<<"container">>, CommandMap}}
-```
-
-`efka` 执行后回复：
-
-```erlang
-{<<"command_response">>, Ref, {<<"container">>, Reply}}
-```
-
-`Reply` 取值：
-
-```erlang
-<<"ok">>
-{<<"ok">>, Result}
-{<<"error">>, Reason}
-```
-
-`Ref` 是 `crypto:strong_rand_bytes(16)` 生成的 16 字节 binary。网络帧只使用 `binary_to_term(PacketBin, [safe])` 可解码的 safe term；协议 label、业务 label、map key 和 action 都使用 binary。
-
-只有 `efka_iot_client` 处于 `activated` 状态时，容器命令才会正常执行；处于非 activated 状态时会返回错误。
-
-## 2. 容器命令 map
-
-`efka_iot_client` 收到网络协议里的 binary-key `CommandMap` 后，直接按 binary key 和 binary action 做函数参数匹配，不再做整包 atom-key 转换。
-
-- command 顶层、`target`、deploy `params`、`create` 中间结构都使用 binary key。
-- `<<"action">>` 的取值使用 binary：`<<"list">>`、`<<"deploy">>`、`<<"start">>`、`<<"stop">>`、`<<"kill">>`、`<<"remove">>`、`<<"config">>`。
-- `undefined` 作为 Erlang 已有 atom 可以直接出现在 safe term 中；协议不再使用额外的哨兵 binary 表示缺省值。
-
-下面各小节描述的是 `efka` 直接接收和处理的 map 格式。
-
-### list
-
-```erlang
-#{<<"action">> => <<"list">>}
-```
-
-执行：
-
-```erlang
-docker_commands:get_containers()
-```
-
-Docker API：
-
-```http
-GET /containers/json?all=true
-```
-
-### deploy
-
-```erlang
-#{
-    <<"action">> => <<"deploy">>,
-    <<"task_id">> => TaskId,
-    <<"params">> => Params
-}
-```
-
-执行：
-
-```erlang
-docker_deploy_manager:deploy(TaskId, Params)
-```
-
-`deploy/2` 会启动独立部署进程，HTTP command response 只表示部署任务是否成功启动。实际部署过程和结果通过 `task_event` 消息流上报给 `iot`。
-
-### start
-
-```erlang
-#{
-    <<"action">> => <<"start">>,
-    <<"target">> => Target
-}
-```
-
-执行：
-
-```erlang
-docker_commands:start_container(ContainerNameOrId)
-```
-
-Docker API：
-
-```http
-POST /containers/{name_or_id}/start
-```
-
-### stop
-
-```erlang
-#{
-    <<"action">> => <<"stop">>,
-    <<"target">> => Target,
-    <<"timeout_seconds">> => TimeoutSeconds
-}
-```
-
-执行：
-
-```erlang
-docker_commands:stop_container(ContainerNameOrId, TimeoutSeconds)
-```
-
-Docker API：
-
-```http
-POST /containers/{name_or_id}/stop?t={TimeoutSeconds}
-```
-
-### kill
-
-```erlang
-#{
-    <<"action">> => <<"kill">>,
-    <<"target">> => Target,
-    <<"signal">> => Signal
-}
-```
-
-执行：
-
-```erlang
-docker_commands:kill_container(ContainerNameOrId, Signal)
-```
-
-### remove
-
-```erlang
-#{
-    <<"action">> => <<"remove">>,
-    <<"target">> => Target,
-    <<"force">> => Force,
-    <<"remove_volumes">> => RemoveVolumes
-}
-```
-
-执行：
-
-```erlang
-docker_commands:remove_container(ContainerNameOrId, Force, RemoveVolumes)
-```
-
-### config
-
-```erlang
-#{
-    <<"action">> => <<"config">>,
-    <<"target">> => Target,
-    <<"config">> => Config
-}
-```
-
-执行：
-
-```erlang
-docker_helper:update_container_config(ContainerNameOrId, iolist_to_binary(Config))
-```
-
-该命令不会调用 Docker API，只会更新 `efka` 主机上对应容器目录里的 `service.conf`。
-
-## 3. Target 解析规则
-
-`target` 是 map：
-
-```erlang
-#{
-    <<"name">> => ContainerName,
-    <<"id">> => ContainerId
-}
-```
-
-解析规则：
-
-- 优先使用 `name`。
-- `name` 为空时使用 `id`。
-- `name` 和 `id` 都为空时会触发匹配错误。
-
-## 4. deploy params 格式
-
-`deploy` 的 `params` 必须包含：
-
-```erlang
-#{
-    <<"container_name">> => ContainerName,
-    <<"create">> => Create
-}
-```
-
-字段说明：
-
-| 字段 | 类型 | 说明 |
-| --- | --- | --- |
-| `container_name` | binary | 容器名称。用于容器创建 URL 的 `name` 参数，也会注入环境变量 `CONTAINER_NAME`。 |
-| `create` | map | Docker create options 的中间结构，由 `iot` 侧构造，`efka` 侧补丁后转为 Docker JSON。 |
-
-`docker_deploy_manager` 会根据 `container_name` 按系统默认规则确保容器目录存在。目录固定为：
-
-```text
-docker.root_dir/container_name/
-```
-
-即使旧客户端在 `params` 中携带 `container_dir`，当前 deploy 流程也会忽略它，避免 HTTP 调用方控制 efka 主机上的写入目录。
-
-## 5. create 中间结构
-
-`create` 的结构：
-
-```erlang
-#{
-    <<"config">> => ContainerConfig,
-    <<"host_config">> => HostConfig,
-    <<"networking_config">> => NetworkingConfig
-}
-```
-
-这三个 map 会被 `docker_container_builder:build_options/3` 转成 Docker Engine API JSON。
-
-## 6. efka 自动补丁规则
-
-在转 Docker JSON 前，`efka` 会先执行补丁：
-
-### 环境变量
-
-向 `create.config.env` 前置注入：
-
-```erlang
-<<"CONTAINER_NAME=", ContainerName/binary>>
-```
-
-如果原 env 列表已经包含完全相同的值，则不会重复添加。
-
-### 配置文件 volume
-
-向 `create.config.volumes` 前置注入容器内配置路径：
-
-```erlang
-<<"/usr/local/etc/service.conf">>
-```
-
-### 配置文件 bind
-
-向 `create.host_config.binds` 前置注入宿主机配置文件映射：
-
-```erlang
-<<ConfigFile/binary, ":/usr/local/etc/service.conf">>
-```
-
-其中 `ConfigFile` 是当前容器目录下的：
-
-```text
-service.conf
-```
-
-如果列表里已存在完全相同的 bind，则不会重复添加。
-
-## 7. Docker JSON 顶层结构
-
-`docker_commands:create_container/2` 最终调用：
-
-```http
-POST /containers/create?name={container_name}
-Content-Type: application/json
-```
-
-请求 body 来自：
-
-```erlang
-Options = docker_container_builder:build_options(ContainerName, ContainerDir, Create),
-Body = iolist_to_binary(json:encode(Options))
-```
-
-最终 JSON 顶层字段：
-
-```json
-{
-  "Image": "...",
-  "Cmd": [],
-  "Entrypoint": [],
-  "Env": [],
-  "Labels": {},
-  "Volumes": {},
-  "User": "",
-  "WorkingDir": "",
-  "Hostname": "",
-  "ExposedPorts": {},
-  "NetworkingConfig": {},
-  "Healthcheck": {},
-  "HostConfig": {}
-}
-```
-
-空列表或空 map 字段有的会保留，有的会被省略到 `{}`，具体见下面映射规则。
-
-## 8. create.config 到 Docker JSON
-
-| 中间字段 | Docker JSON 字段 | 转换规则 |
-| --- | --- | --- |
-| `image` | `Image` | 转成 binary；缺省为 `""`。部署流程拉镜像时会对 image 补 `:latest`，但 create JSON 使用传入值。 |
-| `cmd` | `Cmd` | 列表元素逐个转 binary。 |
-| `entrypoint` | `Entrypoint` | 列表元素逐个转 binary。 |
-| `env` | `Env` | 列表元素逐个转 binary，并自动注入 `CONTAINER_NAME=...`。 |
-| `labels` | `Labels` | key/value 都转 binary。 |
-| `volumes` | `Volumes` | 转成 Docker 要求的 object，key 是容器内路径，value 是 `{}`。 |
-| `user` | `User` | 转成 binary；缺省为 `""`。 |
-| `working_dir` | `WorkingDir` | 转成 binary；缺省为 `""`。 |
-| `hostname` | `Hostname` | 转成 binary；缺省为 `""`。 |
-| `exposed_ports` | `ExposedPorts` | 转成 Docker 端口 object。 |
-| `healthcheck` | `Healthcheck` | 转成 Docker Healthcheck object；未传时为 `{}`。 |
-
-### Volumes
-
-输入：
-
-```erlang
-[<<"/data">>, <<"/usr/local/etc/service.conf">>]
-```
-
-输出 JSON：
-
-```json
-{
-  "Volumes": {
-    "/data": {},
-    "/usr/local/etc/service.conf": {}
-  }
-}
-```
-
-### ExposedPorts
-
-输入：
-
-```erlang
-[
-    #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-    #{<<"container_port">> => 53, <<"protocol">> => <<"udp">>}
-]
-```
-
-输出 JSON：
-
-```json
-{
-  "ExposedPorts": {
-    "80/tcp": {},
-    "53/udp": {}
-  }
-}
-```
-
-规则：
-
-- `protocol` 为空或 `<<"tcp">>` 时输出 `tcp`。
-- 其他 protocol 原样输出。
-
-### Healthcheck
-
-该字段由 iot 校验后下发。`test` 必须是 binary list，`interval_ns`、`timeout_ns` 和 `retries` 必须是非负整数；efka 只按收到的中间 map 转成 Docker JSON 字段。
-
-输入：
-
-```erlang
-#{
-    <<"test">> => [<<"CMD-SHELL">>, <<"curl -f http://localhost || exit 1">>],
-    <<"interval_ns">> => 30000000000,
-    <<"timeout_ns">> => 10000000000,
-    <<"retries">> => 3
-}
-```
-
-输出 JSON：
-
-```json
-{
-  "Healthcheck": {
-    "Test": ["CMD-SHELL", "curl -f http://localhost || exit 1"],
-    "Interval": 30000000000,
-    "Timeout": 10000000000,
-    "Retries": 3
-  }
-}
-```
-
-## 9. create.host_config 到 Docker JSON
-
-`host_config` 会被合并到 Docker JSON 的 `HostConfig` 字段。
-
-| 中间字段 | Docker JSON 字段 | 转换规则 |
-| --- | --- | --- |
-| `binds` | `HostConfig.Binds` | 列表元素逐个转 binary，并自动注入 `service.conf` bind。空列表时省略。 |
-| `network_mode` | `HostConfig.NetworkMode` | 空 binary 时省略。 |
-| `restart_policy` | `HostConfig.RestartPolicy` | 转成 `Name` 和可选 `MaximumRetryCount`。 |
-| `privileged` | `HostConfig.Privileged` | 只有 true 时输出。false 时省略。 |
-| `cap_add` / `cap_drop` | `HostConfig.CapAdd` / `HostConfig.CapDrop` | 两者都为空时省略；只要一个非空，两个字段都会输出。 |
-| `devices` | `HostConfig.Devices` | 转成 Docker device object 列表。 |
-| `memory` | `HostConfig.Memory` | 0 时省略。 |
-| `memory_reservation` | `HostConfig.MemoryReservation` | 0 时省略。 |
-| `nano_cpus` | `HostConfig.NanoCpus` | 0 时省略。 |
-| `cpu_shares` | `HostConfig.CpuShares` | 0 时省略。 |
-| `port_bindings` | `HostConfig.PortBindings` | 转成 Docker 端口绑定 object；空列表时省略。 |
-| `ulimits` | `HostConfig.Ulimits` | 空列表时省略。 |
-| `tmpfs` | `HostConfig.Tmpfs` | 空 map 时省略。 |
-| `sysctls` | `HostConfig.Sysctls` | 空 map 时省略。 |
-| `extra_hosts` | `HostConfig.ExtraHosts` | 空列表时省略。 |
-
-### Binds
-
-输入：
-
-```erlang
-[
-    <<"/host/data:/data">>,
-    <<"/host/log:/var/log:ro">>
-]
-```
-
-efka 自动补丁后输出：
-
-```json
-{
-  "HostConfig": {
-    "Binds": [
-      "/path/to/container/service.conf:/usr/local/etc/service.conf",
-      "/host/data:/data",
-      "/host/log:/var/log:ro"
-    ]
-  }
-}
-```
-
-### RestartPolicy
-
-输入：
-
-```erlang
-#{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "RestartPolicy": {
-      "Name": "always"
-    }
-  }
-}
-```
-
-输入：
-
-```erlang
-#{<<"name">> => <<"on-failure">>, <<"maximum_retry_count">> => 3}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "RestartPolicy": {
-      "Name": "on-failure",
-      "MaximumRetryCount": 3
-    }
-  }
-}
-```
-
-### Devices
-
-输入：
-
-```erlang
-[
-    #{
-        <<"path_on_host">> => <<"/dev/ttyUSB0">>,
-        <<"path_in_container">> => <<"/dev/ttyUSB0">>,
-        <<"cgroup_permissions">> => <<"rwm">>
-    }
-]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Devices": [
-      {
-        "PathOnHost": "/dev/ttyUSB0",
-        "PathInContainer": "/dev/ttyUSB0",
-        "CgroupPermissions": "rwm"
-      }
-    ]
-  }
-}
-```
-
-`cgroup_permissions` 为空时默认输出 `rwm`。
-
-### PortBindings
-
-输入：
-
-```erlang
-[
-    #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>},
-    #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>}
-]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "PortBindings": {
-      "80/tcp": [
-        {"HostIp": "", "HostPort": "8080"}
-      ],
-      "443/tcp": [
-        {"HostIp": "", "HostPort": "443"}
-      ]
-    }
-  }
-}
-```
-
-规则：
-
-- key 使用容器端口和协议组成：`container_port/protocol`。
-- `protocol` 为空或 `tcp` 时输出 `tcp`。
-- `HostPort` 按 Docker API 要求输出为字符串。
-
-### 资源限制
-
-输入：
-
-```erlang
-#{
-    <<"memory">> => 536870912,
-    <<"memory_reservation">> => 268435456,
-    <<"nano_cpus">> => 1500000000,
-    <<"cpu_shares">> => 512
-}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Memory": 536870912,
-    "MemoryReservation": 268435456,
-    "NanoCpus": 1500000000,
-    "CpuShares": 512
-  }
-}
-```
-
-值为 0 的资源字段会被省略。
-
-### Ulimits
-
-输入：
-
-```erlang
-[
-    #{<<"name">> => <<"nofile">>, <<"soft">> => 1024, <<"hard">> => 2048}
-]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Ulimits": [
-      {
-        "Name": "nofile",
-        "Soft": 1024,
-        "Hard": 2048
-      }
-    ]
-  }
-}
-```
-
-### Tmpfs
-
-输入：
-
-```erlang
-#{
-    <<"/tmp">> => <<>>,
-    <<"/run">> => <<"rw,size=64m">>
-}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Tmpfs": {
-      "/tmp": "",
-      "/run": "rw,size=64m"
-    }
-  }
-}
-```
-
-### Sysctls
-
-输入：
-
-```erlang
-#{<<"net.ipv4.ip_forward">> => <<"1">>}
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "Sysctls": {
-      "net.ipv4.ip_forward": "1"
-    }
-  }
-}
-```
-
-### ExtraHosts
-
-输入：
-
-```erlang
-[<<"host.docker.internal:host-gateway">>]
-```
-
-输出：
-
-```json
-{
-  "HostConfig": {
-    "ExtraHosts": ["host.docker.internal:host-gateway"]
-  }
-}
-```
-
-## 10. create.networking_config 到 Docker JSON
-
-输入：
-
-```erlang
-#{
-    <<"endpoints">> => [
-        #{<<"name">> => <<"bridge">>},
-        #{<<"name">> => <<"mynet">>}
-    ]
-}
-```
-
-输出：
-
-```json
-{
-  "NetworkingConfig": {
-    "EndpointsConfig": {
-      "bridge": {},
-      "mynet": {}
-    }
-  }
-}
-```
-
-未传或 endpoints 为空时输出：
-
-```json
-{
-  "NetworkingConfig": {}
-}
-```
-
-## 11. 完整转换示例
-
-收到的 deploy command：
-
-```erlang
-{<<"command">>, Ref, {<<"container">>, #{
-    <<"action">> => <<"deploy">>,
-    <<"task_id">> => 1001,
-    <<"params">> => #{
-        <<"container_name">> => <<"my_nginx">>,
-        <<"create">> => #{
-            <<"config">> => #{
-                <<"image">> => <<"docker.io/library/nginx:latest">>,
-                <<"cmd">> => [<<"nginx">>, <<"-g">>, <<"daemon off;">>],
-                <<"env">> => [<<"ENV=prod">>],
-                <<"volumes">> => [<<"/data">>],
-                <<"exposed_ports">> => [#{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>}]
-            },
-            <<"host_config">> => #{
-                <<"binds">> => [<<"/host/data:/data">>],
-                <<"restart_policy">> => #{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0},
-                <<"memory">> => 536870912,
-                <<"port_bindings">> => [
-                    #{
-                        <<"host_ip">> => <<>>,
-                        <<"host_port">> => 8080,
-                        <<"container_port">> => 80,
-                        <<"protocol">> => <<"tcp">>
-                    }
-                ]
-            },
-            <<"networking_config">> => #{
-                <<"endpoints">> => [#{<<"name">> => <<"bridge">>}]
-            }
-        }
-    }
-}}}
-```
-
-生成的 Docker JSON 示意：
-
-```json
-{
-  "Image": "docker.io/library/nginx:latest",
-  "Cmd": ["nginx", "-g", "daemon off;"],
-  "Entrypoint": [],
-  "Env": ["CONTAINER_NAME=my_nginx", "ENV=prod"],
-  "Labels": {},
-  "Volumes": {
-    "/usr/local/etc/service.conf": {},
-    "/data": {}
-  },
-  "User": "",
-  "WorkingDir": "",
-  "Hostname": "",
-  "ExposedPorts": {
-    "80/tcp": {}
-  },
-  "NetworkingConfig": {
-    "EndpointsConfig": {
-      "bridge": {}
-    }
-  },
-  "Healthcheck": {},
-  "HostConfig": {
-    "Binds": [
-      "/efka/root/my_nginx/service.conf:/usr/local/etc/service.conf",
-      "/host/data:/data"
-    ],
-    "RestartPolicy": {
-      "Name": "always"
-    },
-    "Memory": 536870912,
-    "PortBindings": {
-      "80/tcp": [
-        {
-          "HostIp": "",
-          "HostPort": "8080"
-        }
-      ]
-    }
-  }
-}
-```
-
-## 12. 部署流程补充
-
-deploy action 的执行流程：
-
-1. 根据 `container_name` 和 `docker.root_dir` 确保默认容器目录存在。
-2. 启动独立部署进程。
-3. 部署进程上报任务事件流。
-4. 规范化镜像名：如果镜像最后一段没有 tag，则补 `:latest`。
-5. 调用 Docker API 拉取镜像。
-6. 构造 Docker create JSON。
-7. 调用 `POST /containers/create?name={container_name}`。
-8. 创建空 `service.conf` 文件。
-9. 写入部署摘要日志并关闭任务事件流。
-
-注意：当前 `ensure_container_absent/2` 只上报“开始创建容器”，不会删除已有同名容器；如果 Docker 返回名称冲突，部署会失败并上报“本地容器已经存在”。

docs/efka_iot_protocol.md

@@ -1,320 +0,0 @@
-# EFKA 与 IOT 交互协议
-
-本文档描述 `efka` 与 `iot` 之间的 TLS 长连接协议。当前协议由 Erlang term 直接序列化，发送端使用 `term_to_binary/1`，接收端使用 `binary_to_term(PacketBin, [safe])`。
-
-协议帧只使用 safe external term：顶层 label、业务 label、map key 使用 binary；`Ref` 使用 `crypto:strong_rand_bytes(16)` 生成，是 16 字节 binary。网络协议里不发送 Erlang `reference()`，也不依赖动态创建 atom。
-
-## 传输层
-
-- `efka` 作为 TLS client 连接 `iot`。
-- `iot` 作为 TLS server 接收多个 `efka` 连接，一个连接对应一个 `ssl_channel` 进程。
-- socket 使用 `{packet, 4}`，每个 Erlang term binary 作为一个完整包发送。
-- `Ref` 使用 `crypto:strong_rand_bytes(16)` 生成，只在当前连接的 inflight 表内匹配。
-
-## 顶层帧
-
-协议顶层 tuple 用来表达交互语义：
-
-```erlang
-{<<"request">>, Ref, Body}
-{<<"response">>, Ref, Reply}
-{<<"command">>, Ref, {Domain, Payload}}
-{<<"command_response">>, Ref, {Domain, Reply}}
-{<<"message">>, Body}
-{<<"stream">>, StreamId, Body}
-```
-
-语义说明：
-
-| 帧 | 方向 | 语义 |
-| --- | --- | --- |
-| `{<<"request">>, Ref, Body}` | efka -> iot | efka 发起请求，需要 iot 回复 |
-| `{<<"response">>, Ref, Reply}` | iot -> efka | iot 对 efka request 的回复 |
-| `{<<"command">>, Ref, {Domain, Payload}}` | iot -> efka | iot 下发命令，需要 efka 回复 |
-| `{<<"command_response">>, Ref, {Domain, Reply}}` | efka -> iot | efka 对 iot command 的回复 |
-| `{<<"message">>, Body}` | 双向 | 异步消息，不要求回复 |
-| `{<<"stream">>, StreamId, Body}` | 双向 | 透明 TCP 字节流多路复用 |
-
-`command` 和 `command_response` 的 `Domain` 表示业务域，目前支持：
-
-- `<<"container">>`
-
-## 透明 TCP Stream
-
-`stream` 用于在同一条 TLS 长连接上复用多条透明 TCP 字节流。旧的 `request`、`response`、`command`、`command_response`、`message` 帧格式保持不变；`StreamId = 0` 只作为保留概念，不出现在新的 `stream` 帧里。
-
-`StreamId` 规则：
-
-- `0`：保留给现有控制流。
-- 奇数：`iot` 发起。
-- 偶数：`efka` 发起，当前预留。
-- 同一条 TLS 连接内 `StreamId` 不复用。
-
-`Body` 取值：
-
-```erlang
-<<"open">>
-<<"opened">>
-{<<"open_error">>, Reason}
-{<<"data">>, Chunk}
-<<"fin">>
-{<<"reset">>, Reason}
-```
-
-语义：
-
-- `open`：发起方请求打开一个透明 TCP stream。`open` 不携带参数；`efka` 端固定连接本机 `stream_target`，通常指向本机 manager/nginx 入口。
-- `opened`：本地 TCP 连接建立成功。
-- `open_error`：本地 TCP 连接建立失败，stream 结束。
-- `data`：透明字节数据，`Chunk` 是 binary。HTTP 请求头、请求体、响应头、响应体都只是该 binary 的内容，iot/efka 中间协议不解析 HTTP。
-- `fin`：半关闭，发送方后续不再发送 `data`，但仍可继续接收对端 `data`。
-- `reset`：异常关闭，双方应立即释放该 `StreamId` 的资源。
-
-典型 iot 发起访问 efka 本机监控服务的流程：
-
-```erlang
-iot  -> efka: {<<"stream">>, 1, <<"open">>}
-efka -> iot:  {<<"stream">>, 1, <<"opened">>}
-
-iot  -> efka: {<<"stream">>, 1, {<<"data">>, RawHttpRequestBytes}}
-iot  -> efka: {<<"stream">>, 1, <<"fin">>}
-
-efka -> iot:  {<<"stream">>, 1, {<<"data">>, RawHttpResponseBytes}}
-efka -> iot:  {<<"stream">>, 1, <<"fin">>}
-```
-
-## 鉴权请求
-
-初始连接由 `efka` 发起鉴权 request。每条 TLS 连接只允许一次鉴权；`iot` 侧鉴权成功后会在 `ssl_channel` 标记该连接已鉴权，如果同一连接再次发送 `auth_request`，`iot` 会直接关闭连接。
-
-```erlang
-{<<"request">>, Ref, {<<"auth_request">>, #{
-    <<"uuid">> => UUID,
-    <<"token">> => Token,
-    <<"timestamp">> => Timestamp
-}}}
-```
-
-`iot` 回复：
-
-```erlang
-{<<"response">>, Ref, {<<"auth_response">>, <<"ok">>}}
-{<<"response">>, Ref, {<<"auth_response">>, {<<"error">>, {<<"failed">>, Reason}}}}
-```
-
-处理语义：
-
-- `<<"ok">>`：`efka` 进入 `activated` 状态。
-- `{<<"error">>, {<<"failed">>, Reason}}`：鉴权失败，`iot` 返回失败响应后关闭连接；`efka` 进入重连流程。
-
-## 授权控制
-
-`/host/activate` 只修改 `iot` 本地和持久化的 host 授权状态，不再向 `efka` 下发 auth command。`efka` 可以继续保持连接并发送数据，是否处理这些数据由 `iot_host` 当前状态决定。
-
-因此当前协议没有 `{<<"command">>, Ref, {<<"auth">>, ...}}` 和 `{<<"command_response">>, Ref, {<<"auth">>, ...}}`。授权关闭时，`iot_host` 保持 channel 在线，但不处理上报数据；授权重新打开后，已在线的 channel 可以继续使用。
-
-## 容器管理命令
-
-`iot` 对 `efka` 的容器管理使用 command 语义：
-
-```erlang
-{<<"command">>, Ref, {<<"container">>, CommandMap}}
-```
-
-`efka` 回复：
-
-```erlang
-{<<"command_response">>, Ref, {<<"container">>, Reply}}
-```
-
-`Reply` 取值：
-
-```erlang
-<<"ok">>
-{<<"ok">>, Result}
-{<<"error">>, Reason}
-```
-
-`CommandMap` 使用 binary key 和 binary action；`efka` 接收后直接按 binary key/action 匹配，Docker 参数链路继续使用 binary-key map，不再转换成 atom-key map。
-
-### list
-
-```erlang
-#{<<"action">> => <<"list">>}
-```
-
-返回当前 `efka` 主机上的容器列表。
-
-### deploy
-
-```erlang
-#{
-    <<"action">> => <<"deploy">>,
-    <<"task_id">> => TaskId,
-    <<"params">> => Params
-}
-```
-
-触发容器部署。部署过程中的流式日志不通过该 command response 返回，而是通过 `message` 的 `task_event` 上报。
-
-### start
-
-```erlang
-#{
-    <<"action">> => <<"start">>,
-    <<"target">> => Target
-}
-```
-
-### stop
-
-```erlang
-#{
-    <<"action">> => <<"stop">>,
-    <<"target">> => Target,
-    <<"timeout_seconds">> => TimeoutSeconds
-}
-```
-
-### kill
-
-```erlang
-#{
-    <<"action">> => <<"kill">>,
-    <<"target">> => Target,
-    <<"signal">> => Signal
-}
-```
-
-### remove
-
-```erlang
-#{
-    <<"action">> => <<"remove">>,
-    <<"target">> => Target,
-    <<"force">> => Force,
-    <<"remove_volumes">> => RemoveVolumes
-}
-```
-
-### config
-
-```erlang
-#{
-    <<"action">> => <<"config">>,
-    <<"target">> => Target,
-    <<"config">> => Config
-}
-```
-
-更新容器配置文件。
-
-### Target
-
-容器目标使用 map 表示：
-
-```erlang
-#{
-    <<"name">> => ContainerName,
-    <<"id">> => ContainerId
-}
-```
-
-`name` 和 `id` 至少一个非空；优先使用 `name`，`name` 为空时使用 `id`。
-
-## 异步消息
-
-`message` 不带 `Ref`，不要求对端回复。
-
-### efka -> iot: data
-
-```erlang
-{<<"message">>, {<<"data">>, #{
-    <<"route_key">> => RouteKey,
-    <<"metric">> => Metric
-}}}
-```
-
-用于 `efka` 上报业务指标数据。
-
-### efka -> iot: task_event
-
-```erlang
-{<<"message">>, {<<"task_event">>, #{
-    <<"task_id">> => TaskId,
-    <<"type">> => Type,
-    <<"stream">> => Stream
-}}}
-```
-
-任务事件流关闭时：
-
-```erlang
-{<<"message">>, {<<"task_event">>, #{
-    <<"task_id">> => TaskId,
-    <<"type">> => <<"close">>,
-    <<"stream">> => Reason
-}}}
-```
-
-`task_event` 本身不携带 `uuid`。`iot` 在接收该消息时使用当前已鉴权 `ssl_channel` 绑定的 host UUID，把事件路由到内部任务进程 `{UUID, TaskId}`。HTTP 页面通过 SSE 订阅时也必须使用同一组参数：
-
-```http
-GET /event_stream?uuid=<host_uuid>&task_id=<task_id>
-```
-
-`iot` 会为每个 `{UUID, TaskId}` 维护一个独立的任务进程，用于缓存最近的部署日志、支持多个 SSE listener，并在收到 close 事件后结束事件流。
-
-### efka -> iot: ping
-
-```erlang
-{<<"message">>, <<"ping">>}
-```
-
-用于 TLS 长连接的应用层保活。`efka` 在鉴权成功后周期发送，当前发送间隔为 30 秒。
-
-`iot` 收到后回复：
-
-```erlang
-{<<"message">>, <<"pong">>}
-```
-
-`ping/pong` 只表示 TLS 连接仍可读写，不参与 host online/offline 判定。host 上下线仍由 UDP 心跳和 `iot_host` 本地连接状态共同维护。
-
-### iot -> efka: pub
-
-```erlang
-{<<"message">>, {<<"pub">>, #{
-    <<"topic">> => Topic,
-    <<"qos">> => Qos,
-    <<"content">> => Content
-}}}
-```
-
-用于 `iot` 向 `efka` 本地订阅系统发布 topic 消息。
-
-## 状态与超时
-
-- `efka` 鉴权超时时间：5 秒。
-- `iot` command inflight 超时时间：60 秒。
-- `iot` SSL channel 空闲超时时间：120 秒。120 秒内没有收到任何 TLS 包，包括 `ping`、业务 `message`、`request`、`command_response`，`iot` 会主动关闭该连接。
-- `iot` 管理多个 `efka` 时，每个连接有独立 `ssl_channel` 和独立 inflight 表。
-- command 超时后，`iot` 删除 inflight 记录；之后如果迟到的 `command_response` 到达，会被视为未预期响应。
-
-## UDP 心跳
-
-`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。TLS control channel 和 UDP 心跳共用 `iot_server.host`，分别使用 `tls_port` 和 `udp_port`。UDP 心跳包使用 HMAC-SHA256 校验，HMAC key 为 `SHA256(auth.token)`。
-
-详细格式见 [heartbeat.md](heartbeat.md)。
-
-## 兼容性
-
-当前协议不兼容旧 tuple：
-
-- 旧容器管理：`{request, Ref, {container_request, ...}}`
-- 旧容器回复：`{response, Ref, {container_response, ...}}`
-- 旧授权控制：`{message, {auth_control, Command}}`
-- 已移除的 auth command：`{command, Ref, {auth, activate | deactivate}}`
-- 旧 Ref：Erlang `reference()`，例如 `make_ref()` 生成的值。
-
-如果需要滚动升级，应先增加临时兼容分支或引入协议版本协商。

docs/heartbeat.md

@@ -1,52 +0,0 @@
-# UDP 心跳
-
-`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。该心跳只表示主机存活，不依赖 TLS control channel 是否在线。
-
-## 配置
-
-TLS 和 UDP 共用同一个 `iot_server.host`：
-
-```erlang
-{iot_server, [
-  {host, "localhost"},
-  {tls_port, 443},
-  {udp_port, 18080}
-]},
-
-{heartbeat, [
-  {interval, 5000}
-]},
-
-{auth, [
-  {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"},
-  {token, "zpxlkvmqwnbghytrujsdieofazxcvbnm"}
-]}
-```
-
-- `tls_port` 用于 `efka_iot_client` 建立 TLS 连接。
-- `udp_port` 用于 `efka_iot_heartbeat` 发送 UDP 心跳。
-- `heartbeat.interval` 是发送间隔，单位毫秒，默认 5000。
-- UDP 心跳和 TLS 鉴权复用 `auth.uuid` 和 `auth.token`。
-
-## 包格式
-
-```erlang
-<<
-    Version:8,
-    UuidLen:16,
-    UUID:UuidLen/binary,
-    Timestamp:64/unsigned-big,
-    Nonce:16/binary,
-    Mac:32/binary
->>
-```
-
-`Mac` 使用 HMAC-SHA256：
-
-```erlang
-HeartbeatSecret = crypto:hash(sha256, Token),
-Payload = <<Version:8, UuidLen:16, UUID:UuidLen/binary, Timestamp:64/unsigned-big, Nonce:16/binary>>,
-Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload)
-```
-
-其中 `Token` 是 `auth.token`。`iot` 侧注册 efka client 时保存同样的 `SHA256(Token)` 派生值，并用它校验心跳。

docs/service_websocket_protocol.md

@@ -1,439 +0,0 @@
-# EFKA 与 Service WebSocket 交互协议
-
-本文档描述部署在 `efka` 主机上的 service 与 `efka` 之间的 WebSocket 通讯逻辑。当前接入实现位于：
-
-- [efka_service_channel.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service_channel.erl)
-- [service.proto](/usr/local/code/cloudkit/efka/apps/efka/proto/service.proto)
-- [efka_subscription.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_subscription.erl)
-- [efka_service.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service.erl)
-
-## 1. 连接入口
-
-`efka` 启动时会创建 Cowboy WebSocket server，监听配置来自 `efka.config` 的 `websocket_server`：
-
-```erlang
-{websocket_server, [
-    {port, 18080},
-    {acceptors, 10},
-    {max_connections, 1024},
-    {backlog, 1024}
-]}
-```
-
-WebSocket 路由固定为：
-
-```http
-GET /ws
-```
-
-连接建立后，每个 WebSocket 连接对应一个 `efka_service_channel` 进程。
-
-当前协议没有独立的认证字段。service 必须先发送 `register` request 完成注册；未注册前只能处理 `register`，其它 request 会返回 `invalid request`，cast 会被忽略。
-
-Cowboy WebSocket ping 帧由 `efka_service_channel` 直接回复 pong：
-
-```erlang
-websocket_handle(ping, State) -> {reply, pong, State}
-```
-
-## 2. 二进制帧格式
-
-service 与 `efka` 之间使用 WebSocket binary frame。每个业务 frame 的第 1 个字节是自定义帧类型，剩余字节是 protobuf payload。
-
-```text
-<frame_type:1 byte><protobuf_payload:N bytes>
-```
-
-帧类型：
-
-| frame_type | 十六进制 | 方向 | 语义 | protobuf 类型 |
-| --- | --- | --- | --- | --- |
-| `REQUEST` | `0x01` | service -> efka | 请求，需要 efka 回复 | `ServiceRequest` |
-| `REPLY` | `0x02` | efka -> service | request 的响应 | `ServiceReply` |
-| `CAST` | `0x03` | 双向 | 单向消息，不要求回复 | `ServiceCast` |
-
-`efka_service_channel` 当前只处理 service 发来的 `REQUEST` 和 `CAST`。收到未知格式时只记录日志，不主动发送错误响应。
-
-## 3. Protobuf 定义
-
-协议定义来自 `apps/efka/proto/service.proto`：
-
-```protobuf
-syntax = "proto3";
-
-message ServiceRequest {
-  uint32 packet_id = 1;
-
-  message Register {
-    string service_id = 1;
-  }
-
-  message Subscribe {
-    string topic = 1;
-  }
-
-  oneof request {
-    Register register = 10;
-    Subscribe subscribe = 11;
-  }
-}
-
-message ServiceReply {
-  uint32 packet_id = 1;
-
-  message Error {
-    int32 code = 1;
-    string message = 2;
-  }
-
-  oneof reply {
-    bytes result = 10;
-    Error error = 11;
-  }
-}
-
-message ServiceCast {
-  message MetricData {
-    bytes route_key = 1;
-    bytes metric = 2;
-  }
-
-  message TopicEvent {
-    string topic = 1;
-    bytes content = 2;
-  }
-
-  oneof body {
-    TopicEvent topic_event = 10;
-    MetricData metric_data = 11;
-  }
-}
-```
-
-注意：
-
-- `ServiceRequest.packet_id` 由 service 生成，用于匹配 request 和 reply。
-- `ServiceReply.packet_id` 必须等于原 request 的 `packet_id`。
-- `ServiceReply.result` 当前成功值一般为 `<<"ok">>`。
-- `ServiceReply.error.code` 当前失败时统一使用 `-1`。
-- `MetricData.route_key` 和 `MetricData.metric` 是 bytes。
-- `TopicEvent.topic` 是 string，`TopicEvent.content` 是 bytes。
-
-## 4. request/reply 语义
-
-### 4.1 register
-
-service 建立 WebSocket 后应先发送注册请求：
-
-```protobuf
-ServiceRequest {
-  packet_id: 1
-  register {
-    service_id: "service-a"
-  }
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x01 ++ protobuf(ServiceRequest)
-```
-
-`efka` 处理逻辑：
-
-1. `efka_service_channel` 解码 `ServiceRequest`。
-2. 调用 `efka_service_sup:start_service(ServiceId)` 启动或复用 service 进程。
-3. 调用 `efka_service:attach_channel(ServicePid, ChannelPid)` 绑定当前 WebSocket channel。
-4. 通过 `efka_service_model:insert/1` 写入 service 记录：
-
-```erlang
-#service{
-    service_id = ServiceId,
-    container_name = <<>>,
-    status = ?SERVICE_RUNNING,
-    meta_data = #{},
-    create_ts = efka_util:timestamp(),
-    update_ts = efka_util:timestamp()
-}
-```
-
-如果该 `service_id` 是新记录，`status` 会写入 `?SERVICE_RUNNING`。如果该 `service_id` 已存在，当前 `efka_service_model:insert/1` 只更新 `meta_data`、`container_name` 和 `update_ts`，不会覆盖已有 `status`。
-
-5. `efka_service_channel` 状态更新为：
-
-```erlang
-#state{
-    service_id = ServiceId,
-    service_pid = ServicePid,
-    is_registered = true
-}
-```
-
-成功响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 1
-  result: "ok"
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x02 ++ protobuf(ServiceReply)
-```
-
-失败响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 1
-  error {
-    code: -1
-    message: "attach channel failed"
-  }
-}
-```
-
-常见失败场景：
-
-- 同一个 `service_id` 已有存活 channel，`efka_service:attach_channel/2` 返回 `{error, <<"channel exists">>}`。
-- 当前实现对外统一返回 `"attach channel failed"`，详细原因只写日志。
-
-### 4.2 subscribe
-
-注册成功后，service 可以订阅 topic：
-
-```protobuf
-ServiceRequest {
-  packet_id: 2
-  subscribe {
-    topic: "device/*/event"
-  }
-}
-```
-
-`efka` 处理逻辑：
-
-1. 要求当前 channel 已注册，即 `is_registered = true`。
-2. 调用 `efka_subscription:subscribe(Topic, ChannelPid)`。
-3. 订阅成功后，把 topic 加入 `efka_service_channel` 的 `subscribed_topics` 集合。
-
-成功响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 2
-  result: "ok"
-}
-```
-
-失败响应：
-
-```protobuf
-ServiceReply {
-  packet_id: 2
-  error {
-    code: -1
-    message: "invalid topic name"
-  }
-}
-```
-
-未注册时发送 subscribe 会走通用错误：
-
-```protobuf
-ServiceReply {
-  packet_id: 2
-  error {
-    code: -1
-    message: "invalid request"
-  }
-}
-```
-
-## 5. cast 语义
-
-### 5.1 service -> efka: metric_data
-
-service 通过 `CAST` 上报指标数据：
-
-```protobuf
-ServiceCast {
-  metric_data {
-    route_key: "device/a/temp"
-    metric: "{\"value\":23.5}"
-  }
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x03 ++ protobuf(ServiceCast)
-```
-
-`efka` 处理逻辑：
-
-1. `efka_service_channel` 解码 `ServiceCast`。
-2. 要求当前 channel 已注册。
-3. 调用：
-
-```erlang
-efka_service:metric_data(ServicePid, RouteKey, Metric)
-```
-
-4. `efka_service` 再调用：
-
-```erlang
-efka_iot_client:metric_data(RouteKey, Metric)
-```
-
-5. `efka_iot_client` 通过 efka 与 iot 的 TLS 长连接把数据上报给 iot：
-
-```erlang
-{<<"message">>, {<<"data">>, #{
-    <<"route_key">> => RouteKey,
-    <<"metric">> => Metric
-}}}
-```
-
-`metric_data` 是单向消息，service 不会收到 reply。未注册或未知 cast body 会被忽略。
-
-### 5.2 efka -> service: topic_event
-
-当 `iot` 通过 efka 与 iot 的 TLS channel 下发 pub 消息到 `efka` 时，`efka_iot_client` 会调用：
-
-```erlang
-efka_subscription:publish(Topic, Qos, Content)
-```
-
-`efka_subscription` 根据 topic 匹配已订阅的 `efka_service_channel`。匹配成功后向 channel 进程发送：
-
-```erlang
-{topic_broadcast, Topic, Content}
-```
-
-`efka_service_channel` 再推送 WebSocket `CAST` 给 service：
-
-```protobuf
-ServiceCast {
-  topic_event {
-    topic: "device/a/event"
-    content: "payload bytes"
-  }
-}
-```
-
-WebSocket binary frame：
-
-```text
-0x03 ++ protobuf(ServiceCast)
-```
-
-`topic_event` 是单向消息，不要求 service 回复。
-
-## 6. topic 匹配规则
-
-`efka_subscription` 当前使用 `/` 拆分 topic：
-
-```erlang
-binary:split(Topic, <<$/>>, [global])
-```
-
-支持两种通配符：
-
-| 通配符 | 语义 |
-| --- | --- |
-| `*` | 单级匹配，只匹配一个 segment |
-| `+` | 多级匹配，只能出现在最后一段，且至少匹配一个剩余 segment |
-
-示例：
-
-| 订阅 topic | 发布 topic | 是否匹配 |
-| --- | --- | --- |
-| `device/a/temp` | `device/a/temp` | 是 |
-| `device/*/temp` | `device/a/temp` | 是 |
-| `device/*/temp` | `device/a/b/temp` | 否 |
-| `device/+` | `device/a` | 是 |
-| `device/+` | `device/a/temp` | 是 |
-| `device/+` | `device` | 否 |
-
-同一个 `efka_service_channel` 订阅多个 topic 时，同一次 publish 最多投递一次。
-
-## 7. QoS 与遗留消息
-
-`efka_subscription:publish(Topic, Qos, Content)` 中的 `Qos` 影响没有订阅者时的行为：
-
-- `Qos = 0`：没有匹配订阅者时直接丢弃。
-- `Qos /= 0`：没有匹配订阅者时暂存到 `remaining_messages`。
-
-当后续有 service 订阅能匹配这些 topic 时，`efka_subscription` 会把对应遗留消息推送给该 service，并从 `remaining_messages` 移除。
-
-当前 `remaining_messages` 存在内存中，不持久化；`efka` 重启后会丢失。
-
-## 8. 生命周期与清理
-
-### WebSocket channel 关闭
-
-`efka_service_channel:terminate/3` 会执行：
-
-1. 调用 `efka_subscription:unsubscribe_all(ChannelPid)` 清理该 channel 的所有订阅。
-2. 如果 channel 已完成 register，则调用：
-
-```erlang
-efka_service_model:change_status(ServiceId, ?SERVICE_STOPPED)
-```
-
-也就是把 service 状态更新为停止。
-
-### service 进程退出
-
-`efka_service_channel` monitor 绑定的 `efka_service` 进程。如果 service 进程退出，channel 会停止：
-
-```erlang
-websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State) ->
-    {stop, State#state{service_pid = undefined}}
-```
-
-`efka_service` 同时也 monitor channel。channel 退出时，`efka_service` 会清空自身的 `channel_pid`。
-
-## 9. 完整交互流程
-
-典型流程：
-
-```text
-service                         efka/efka_service_channel              efka_service              efka_subscription
-  |                                      |                              |                            |
-  |-- WebSocket connect /ws ------------>|                              |                            |
-  |                                      |                              |                            |
-  |-- REQUEST register ----------------->|                              |                            |
-  |                                      |-- start_service(service_id) ->|                            |
-  |                                      |-- attach_channel ----------->|                            |
-  |<------------- REPLY result="ok" -----|                              |                            |
-  |                                      |                              |                            |
-  |-- REQUEST subscribe ---------------->|                              |                            |
-  |                                      |---------------- subscribe ------------------------------>|
-  |<------------- REPLY result="ok" -----|                              |                            |
-  |                                      |                              |                            |
-  |-- CAST metric_data ----------------->|                              |                            |
-  |                                      |-- metric_data -------------->|                            |
-  |                                      |                              |-- efka_iot_client:metric_data -> iot
-  |                                      |                              |                            |
-  |<------------- CAST topic_event ------|<--------------- topic_broadcast -----------------------|
-  |                                      |                              |                            |
-  |-- WebSocket close ------------------>|                              |                            |
-  |                                      |-- unsubscribe_all -------------------------------------->|
-  |                                      |-- change_status(service_id, stopped)
-```
-
-## 10. 当前限制
-
-- 没有认证字段，service 只通过 `service_id` 注册。
-- `register` 可重复连接同一个 `service_id`，但同一时间只允许一个活跃 channel 绑定到 `efka_service`。
-- request 只支持 `register` 和 `subscribe`。
-- cast 只支持 `metric_data` 和 `topic_event`。
-- `subscribed_topics` 目前只在 `efka_service_channel` 内记录，关闭时实际清理由 `efka_subscription:unsubscribe_all/1` 完成。
-- `efka_subscription` 当前基于列表扫描匹配，订阅规模很大时需要后续优化为索引结构。

docs/todo.md

@@ -1,50 +0,0 @@
-# TODO
-
-## 协议层
-
-### 高优先级
-
-- 将 `message.proto` 收敛为 `efka` 与 `iot` 共用的一份单一事实来源，避免协议定义漂移和双端重复维护。
-
-### 中优先级
-
-- 增强 protobuf 映射和 Docker JSON 生成相关的自动化契约测试。
-
-## Docker 层
-
-### 高优先级
-
-- 将 `docker_commands` 拆分为更清晰的层次：Docker HTTP 客户端、Docker JSON 构造层、以及 efka 本地补丁逻辑。
-- 明确 `docker_task_reporter` 的投递语义；当前只有内存缓冲，还缺少持久化能力和队列上限控制。
-
-### 中优先级
-
-- 丰富 `docker_deploy_manager` 的任务跟踪状态，把部署元数据、耗时、阶段、失败原因等信息纳入统一管理，避免后续再回头重构。
-- 统一 `docker_container_service` 的返回结构，避免调用方分支处理 `ok | {ok, binary()} | {error, binary()}` 这种混合形式。
-- 消除 `docker_commands` 中重复的 Docker HTTP 响应解析逻辑。
-
-### 低优先级
-
-- 对于像 `docker_events.erl` 这样当前未启用的模块，要么清理掉，要么补充说明其保留原因。
-
-## 订阅与通道层
-
-### 高优先级
-
-- 将 `efka_subscription` 当前基于列表扫描的匹配方式替换为更可扩展的索引结构，例如 trie 或基于 ETS 的索引。
-
-### 中优先级
-
-- 继续收敛 `efka_iot_client` 的职责，将传输层状态管理与 request/cast 协议处理进一步拆开。
-- 评估并落实 `efka_service_channel` 中 `subscribed_topics` 集合的用途；如果只是被动保存状态，则应简化。
-
-### 低优先级
-
-- 如果 `efka_iot_client` 的缓存指标量继续增长，可以为缓存刷出增加批量发送或节流机制。
-
-## 基础设施层
-
-### 低优先级
-
-- 如果上传相关能力已经不再属于当前运行时，移除或补充说明残留的启动逻辑，例如 `ensure_upload_dir/0`。
-- 评估是否将 `efka_logger` 与 OTP `logger` 统一，避免长期维护两套日志链路。

env.file

@@ -1,15 +0,0 @@
-[dev]
-EFKA_DETS_DIR="/usr/local/var/lib/efka/dets/"
-EFKA_IOT_HOST="localhost"
-EFKA_AUTH_UUID="qbxmjyzrkpntfgswaevodhluicqzxplkm"
-EFKA_AUTH_TOKEN="zpxlkvmqwnbghytrujsdieofazxcvbnm"
-EFKA_DOCKER_ROOT_DIR="/usr/local/var/lib/efka/docker/"
-EFKA_MNESIA_DIR="/usr/local/var/lib/efka/mnesia"
-
-[prod]
-EFKA_DETS_DIR="/var/lib/efka/dets/"
-EFKA_IOT_HOST="localhost"
-EFKA_AUTH_UUID="qbxmjyzrkpntfgswaevodhluicqzxplkm"
-EFKA_AUTH_TOKEN="zpxlkvmqwnbghytrujsdieofazxcvbnm"
-EFKA_DOCKER_ROOT_DIR="/var/lib/efka/docker/"
-EFKA_MNESIA_DIR="/var/lib/efka/mnesia"

rebar.config

@@ -1,15 +1,11 @@
 {erl_opts, [debug_info]}.
-
-{project_app_dirs, ["apps/*"]}.
-
-{plugins, [
-    {rebar3_gpb_plugin, ".*", {git, "https://github.com/lrascao/rebar3_gpb_plugin.git", {tag, "2.23.8"}}}
-]}.
-
 {deps, [
     {sync, ".*", {git, "https://github.com/rustyio/sync.git", {branch, "master"}}},
+    {jiffy, ".*", {git, "https://github.com/davisp/jiffy.git", {tag, "1.1.2"}}},
     {cowboy, ".*", {git, "https://github.com/ninenines/cowboy.git", {tag, "2.10.0"}}},
-    {gun, ".*", {git, "https://github.com/ninenines/gun.git", {tag, "2.2.0"}}}
+    {gun, ".*", {git, "https://github.com/ninenines/gun.git", {tag, "2.2.0"}}},
+    {parse_trans, ".*", {git, "https://github.com/uwiger/parse_trans", {tag, "3.0.0"}}},
+    {lager, ".*", {git,"https://github.com/erlang-lager/lager.git", {tag, "3.9.2"}}}
 ]}.
 
 {relx, [{release, {efka, "0.1.0"},
@@ -17,10 +13,18 @@
           sasl]},
 
         {mode, dev},
-        {include_erts, false},
 
-        {sys_config_src, "./config/sys.config.src"},
-        {vm_args_src, "./config/vm.args.src"}
+        %% automatically picked up if the files
+        %% exist but can be set manually, which
+        %% is required if the names aren't exactly
+        %% sys.config and vm.args
+        {sys_config, "./config/sys.config"},
+        {vm_args, "./config/vm.args"}
+
+        %% the .src form of the configuration files do
+        %% not require setting RELX_REPLACE_OS_VARS
+        %% {sys_config_src, "./config/sys.config.src"},
+        %% {vm_args_src, "./config/vm.args.src"}
 ]}.
 
 {profiles, [{prod, [{relx,
@@ -34,4 +38,11 @@
                      ]
             }]}]}.
 
-{rebar_packages_cdn, "https://hexpm.upyun.com"}.
+{project_plugins, [
+    %% 或从 Git 仓库拉取最新版本
+    {pc, {git, "https://github.com/blt/port_compiler.git", {tag, "v1.15.0"}}}
+]}.
+
+{erl_opts, [{parse_transform,lager_transform}]}.
+
+{rebar_packages_cdn, "https://hexpm.upyun.com"}.

run

@@ -1,10 +1,6 @@
 #! /bin/sh
 
-export EFKA_DETS_DIR="/usr/local/var/lib/efka/dets/"
-export EFKA_IOT_HOST="localhost"
-export EFKA_AUTH_UUID="qbxmjyzrkpntfgswaevodhluicqzxplkm"
-export EFKA_AUTH_TOKEN="zpxlkvmqwnbghytrujsdieofazxcvbnm"
-export EFKA_DOCKER_ROOT_DIR="/usr/local/var/lib/efka/docker/"
-export EFKA_MNESIA_DIR="/usr/local/var/lib/efka/mnesia"
+rebar3 compile
+rebar3 release
 
 _build/default/rel/efka/bin/efka console