diff --git a/CODE_LOGIC_OVERVIEW.md b/CODE_LOGIC_OVERVIEW.md new file mode 100644 index 0000000..c8b88bc --- /dev/null +++ b/CODE_LOGIC_OVERVIEW.md @@ -0,0 +1,288 @@ +# EFKA 当前代码逻辑梳理 + +> 本文档记录当前阅读代码后对项目逻辑的理解,主要基于 `apps/efka/src/`、`apps/efka/include/`、`config/` 和 protobuf 生成文件。尚未覆盖每个边界条件和全部测试行为。 + +## 1. 项目定位 + +这个项目是一个 Erlang/OTP 应用,核心作用像一个边缘侧 agent: + +- 对本机微服务开放 WebSocket 接入,微服务注册、订阅 topic、上报指标。 +- 与上游服务器建立 TLS 长连接,完成鉴权、上报数据、接收远程命令。 +- 通过 Docker API 管理本机容器,包括部署、启动、停止、杀掉、删除、更新配置和列表查询。 +- 用 DETS 做少量本地持久化,包括服务状态和离线缓存包。 + +依赖主要有: + +- `cowboy`:本地 WebSocket server。 +- `gun`:通过 Unix Socket 访问 Docker HTTP API。 +- `ssl`:连接上游 TLS server。 +- OTP `json` 模块:JSON 编解码。 +- `gpb`:protobuf 代码生成。 + +## 2. 启动流程 + +应用入口是 `efka_app:start/2`。 + +启动步骤: + +1. 设置 Erlang IO 编码为 unicode。 +2. 设置 `fullsweep_after`,试图加快进程内存回收。 +3. 启动 Cowboy WebSocket server。 +4. 启动顶层 supervisor `efka_sup`。 + +WebSocket server: + +- 从 `config/sys.config` 读取 `efka.websocket_server`。 +- 监听 `/ws`。 +- 请求处理模块是 `efka_service_channel`。 + +`efka_sup` 启动的子进程: + +- `efka_logger`:部署日志落盘。 +- `efka_service_sup`:动态管理每个已注册微服务对应的 `efka_service` 进程。 +- `efka_service_model`:DETS 服务状态表。 +- `efka_subscription`:本地 topic 订阅中心。 +- `efka_iot_client`:连接上游 TLS server 的状态机。 + +`docker` application 启动的子进程: + +- `docker_task_reporter`:部署任务事件缓冲和转发。 +- `docker_deploy_manager`:部署任务管理器。 + +`docker_events` 代码存在于 `apps/docker`,但目前默认不会启动。 + +## 3. 两条通信链路 + +项目里有两条主要通信链路。 + +### 3.1 微服务到 EFKA:WebSocket + +本机微服务连接 `ws://:18080/ws`。 + +协议处理在 `efka_service_channel`: + +- WebSocket 收到 `ping` 直接回复 `pong`。 +- 收到 binary 帧时,第一个字节是帧类型。 +- `FRAME_REQUEST = 0x01`,用 `efka_service_pb` 解码为 `ServiceRequest`。 +- `FRAME_CAST = 0x03`,用 `efka_service_pb` 解码为 `ServiceCast`。 +- 回复时使用 `FRAME_REPLY = 0x02`。 + +支持的微服务请求: + +- `register`:注册服务。 +- `subscribe`:订阅 topic。 + +支持的微服务 cast: + +- `metric_data`:上报指标数据。 + +注册流程: + +1. 微服务发送 `ServiceRequest.Register`,包含 `service_id`。 +2. `efka_service_channel` 调用 `efka_service_sup:start_service(ServiceId)`。 +3. 如果该服务进程不存在,则动态启动一个 `efka_service`;如果已存在,复用已有 pid。 +4. `efka_service_channel` 调用 `efka_service:attach_channel(ServicePid, self())` 绑定当前 WebSocket channel。 +5. `efka_service` 只允许绑定一个存活 channel;已有 channel 存活时返回 `channel exists`。 +6. 注册成功后写入 `efka_service_model`,服务状态设为 running。 +7. channel 退出时,取消全部订阅,并把服务状态改为 stopped。 + +指标上报流程: + +1. 微服务发送 `ServiceCast.MetricData`。 +2. `efka_service_channel` 调用 `efka_service:metric_data(ServicePid, RouteKey, Metric)`。 +3. `efka_service` 转发给 `efka_iot_client:metric_data(RouteKey, Metric)`。 +4. `efka_iot_client` 如果处于 activated 状态,直接发给上游;否则写入持久化 outbox。 + +### 3.2 EFKA 到上游:TLS 长连接 + +上游连接逻辑在 `efka_iot_client`,它是一个 `gen_statem`。 + +状态包括: + +- `disconnected`:未连接。 +- `auth`:已连接,等待鉴权响应。 +- `restricted`:鉴权受限,不能正常推送数据,但可以接收部分授权命令。 +- `activated`:已激活,可正常收发。 + +连接流程: + +1. 初始化后立即触发 `create_transport`。 +2. 读取 `efka.tls_server_address`,用 `ssl:connect/4` 建立 TLS 连接。 +3. TLS socket 使用 `{packet, 4}` 和 `{active, true}`。 +4. 连接成功后发送 `AuthRequest`。 +5. 收到鉴权成功 reply 后进入 `activated`,并触发 outbox 刷出。 +6. 连接或鉴权失败时关闭 socket,5 秒后重连。 + +上游协议同样用第一个字节区分帧类型: + +- `FRAME_REQUEST = 0x01` +- `FRAME_REPLY = 0x02` +- `FRAME_CAST = 0x03` + +但 protobuf 使用的是 `message_pb`,不是微服务侧的 `efka_service_pb`。 + +`efka_iot_client` 上报的内容: + +- `metric_data`:业务指标数据,activated 时实时发送,否则进入持久化 outbox。 +- `task_event_stream`:Docker 部署任务流式日志,只在 activated 时发送。 +- `close_task_event_stream`:任务结束事件,只在 activated 时发送。 + +`efka_iot_client` 接收的内容: + +- `RequestFrame.container_request`:远程容器管理请求,交给 `docker_container_service`。 +- `CastFrame.command`:授权命令,`COMMAND_AUTH` 用于切换鉴权/受限状态。 +- `CastFrame.pub`:上游发布的 topic 消息,交给 `efka_subscription:publish/3`。 + +## 4. 本地 Pub/Sub 逻辑 + +`efka_subscription` 是本地订阅中心。 + +订阅: + +- `efka_service_channel` 收到微服务 `subscribe` 后调用 `efka_subscription:subscribe(Topic, self())`。 +- 同一个 pid 对同一个 topic 重复订阅会被忽略。 +- 首次看到某个 subscriber pid 时,会 monitor 该 pid;pid 退出后自动移除订阅。 + +topic 匹配规则: + +- `/` 分隔 topic components。 +- `*` 表示单级匹配。 +- `+` 表示多级匹配,但只能出现在末尾。 +- 完全匹配优先级最高,不过当前代码里的 `order` 字段只是计算并保存,没有实际排序使用。 + +发布: + +- 上游发来的 `CastFrame.Pub` 会调用 `efka_subscription:publish(Topic, Qos, Content)`。 +- 如果匹配到订阅者,向对应 `efka_service_channel` 发送 `{topic_broadcast, Topic, Content}`。 +- `efka_service_channel` 再编码为 `ServiceCast.TopicEvent` 推送给微服务。 +- 如果没有订阅者且 `Qos != 0`,消息会暂存在 `remaining_messages`。 +- 新订阅建立时,会尝试把匹配的 remaining messages 补发给该订阅者。 + +## 5. Docker 容器管理链路 + +上游发来的容器请求最终进入 `docker_container_service:handle_request/1`。 + +支持动作: + +- `list`:列出容器。 +- `deploy`:部署容器。 +- `start`:启动容器。 +- `stop`:停止容器。 +- `kill`:发送 kill 信号。 +- `remove`:删除容器。 +- `config`:更新容器配置文件。 + +简单操作: + +- `start/stop/kill/remove/list` 直接调用 `docker_commands`。 +- `docker_commands` 通过短生命周期普通 `docker_client` 进程访问 `/var/run/docker.sock`。 +- HTTP 客户端用 `gun:open_unix/2`。 +- 流式请求使用 `{docker_client, Ref, Event}` 消息返回,事件包括 `{response, Status, Headers}`、`{data, Bin}`、`done` 和 `{error, Reason}`。 + +部署操作: + +1. `docker_container_service` 收到 `deploy` 后调用 `docker_deploy_manager:deploy(TaskId, Params)`。 +2. `docker_deploy_manager` 根据 `docker.root_dir` 和参数里的 `container_name/container_dir` 确保容器目录存在。 +3. 它启动一个独立 `docker_deployer` 进程,并 monitor 该部署进程。 +4. `docker_deployer` 执行实际部署步骤。 +5. 部署过程事件写入 `docker_task_reporter`。 +6. `docker_task_reporter` 在上游连接 activated 时把事件转给 `efka_iot_client`;未激活时保留队列并定时重试。 + +`docker_deployer` 当前部署步骤: + +1. 上报“开始部署容器”。 +2. 调用 `ensure_container_absent/2`,但当前实现只是上报“开始创建容器”,没有真正删除旧容器。 +3. 规范化镜像名,没有 tag 时补 `:latest`。 +4. 调用 Docker API 拉取镜像,`docker_deployer` 按 `Ref` 接收 `docker_client` 流式消息并上报拉取过程。 +5. 创建容器。 +6. 创建空的 `service.conf` 配置文件。 +7. 写部署摘要到 `efka_logger`。 +8. 关闭任务事件流,状态为 `success` 或 `fail`。 + +创建容器参数: + +- `docker_container_builder` 把 protobuf 的 `DockerCreateOptions` 转成 Docker API JSON。 +- 会自动给容器注入环境变量 `CONTAINER_NAME=`。 +- 会自动添加容器内配置路径 `/usr/local/etc/service.conf`。 +- 会自动把宿主机 `service.conf` bind 到容器内 `/usr/local/etc/service.conf`。 + +配置更新: + +- `docker_container_service:update_container_config/2` 根据容器名找到目录。 +- 写入该目录下的 `service.conf`。 +- 容器目录映射由 `docker_helper` 通过 `.container_dir` 指针文件维护。 + +## 6. 本地持久化 + +### 6.1 `efka_service_model` + +使用 DETS 表 `service`。 + +文件位置来自 `efka.dets_dir`,默认配置是: + +```erlang +"/usr/local/code/tmp/dets/service.dets" +``` + +记录结构在 `apps/efka/include/efka_tables.hrl`: + +- `service_id` +- `container_name` +- `meta_data` +- `status` +- `create_ts` +- `update_ts` + +用途: + +- 注册成功时插入或更新服务。 +- channel 关闭时把服务状态改成 stopped。 +- 支持查询所有服务、运行中服务和单个服务状态。 + +### 6.2 `efka_iot_outbox` + +使用 append-only log 文件和 metadata 文件。 + +作用: + +- 当 `efka_iot_client` 不在 activated 状态时,把待上报 packet 缓存下来。 +- `efka_iot_client` 激活后循环 `next -> send -> ack` 刷出。 +- 所有记录 ack 后会截断 log,下一轮从 seq 1 重新开始。 + +## 7. 日志 + +有两套日志: + +- OTP logger:在 `config/sys.config` 里配置到 console 和 `log/debug.log`。 +- `efka_logger`:自定义部署日志,按日期写到 `code:root_dir() ++ "/log/"` 下。 + +`efka_logger` 主要用于部署任务摘要和失败原因。 + +## 8. 配置项 + +关键配置来自 `config/sys.config`: + +- `docker.root_dir`:容器相关目录根路径。 +- `dets_dir`:DETS 文件目录。 +- `websocket_server`:本地 WebSocket 监听配置。 +- `tls_server_address`:上游 TLS server 地址。 +- `auth`:上游鉴权信息。 + +注意: + +- `efka_service_model` 打开 DETS 前假设 `dets_dir` 已存在,代码里没有显式创建目录。 +- `docker_client` 固定使用 `/var/run/docker.sock`,每次请求内部由短生命周期普通进程执行 open/request/close。 + +## 9. 当前看到的几个注意点 + +- `README.md` 描述的是 JSON-RPC 风格 WebSocket,但当前代码实际处理的是 binary protobuf 帧;README 可能已经过期。 +- `efka_service_channel` 里 `register` 只使用 `service_id`,没有使用 README 中提到的 `meta_data/container_name`。 +- `efka_iot_client:send_result_reply/3` 和 `send_error_reply/3` 编码 `ReplyFrame` 后没有加 `FRAME_REPLY` 前缀;接收侧是否期望裸 protobuf 需要确认。 +- `docker_deployer:ensure_container_absent/2` 当前没有真正确保旧容器不存在,只是上报日志。 +- `efka_subscription` 计算了 topic `order`,但匹配广播时没有使用优先级排序。 +- `docker_commands` 部分错误响应解码没有统一使用 `[return_maps]`,有些分支可能匹配不到 map。 +- `docker_events` 存在但未启动,且使用 shell 命令 `docker events`,与其他 Docker API 访问方式不同。 + +## 10. 一句话主流程 + +微服务通过 WebSocket 注册到 EFKA,本地 channel 把指标交给对应 `efka_service`,再由 `efka_iot_client` 通过 TLS 上报给上游;上游通过同一条 TLS 连接下发 pub/sub 消息和容器管理请求,pub/sub 再广播回本地微服务,容器请求则通过 Docker Unix Socket 在本机执行。 diff --git a/README.md b/README.md index 29d3de6..089a028 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,110 @@ -efka -===== +# ws_channel 模块 API 文档与交互逻辑 -An OTP application +## 注意websocket的数据格式为: text -1. 先解决数据的上行问题 -2. todo list - 要解决连接断开重新连接的问题 !!! +## 一、模块概述 +`ws_channel` 是基于 Erlang + Cowboy WebSocket 实现的 MQTT 相关交互模块,主要用于服务注册、主题订阅、指标数据上报、事件发送及消息广播等功能,通过 WebSocket 协议实现客户端与服务端的实时双向通信。 -Build ------ - $ rebar3 compile +## 三、核心 API 方法 +客户端通过发送 **JSON 格式文本消息** 与服务端交互,消息格式遵循 JSON-RPC 风格(包含 `id`、`method`、`params` 字段)。 + + +### 1. 服务注册(register) +#### 功能 +注册服务并建立客户端与服务进程的关联,是后续操作(订阅、上报数据等)的前提。 + +#### 请求格式 +```json +{ + "id": <整数,请求唯一标识>, + "method": "register", + "params": { + "service_id": <二进制,服务唯一标识,必填>, + "meta_data": <映射,服务元数据,可选>, + "container_name": <二进制,容器名称,可选> + } +} +``` + +#### 响应格式 +- 成功响应: + ```json + { + "id": <与请求id一致>, + "result": "ok" + } + ``` +- 失败处理:服务端直接关闭连接(因 `attach_channel` 失败) + +### 2. 主题订阅(subscribe) +#### 功能 +订阅指定主题,后续可接收该主题的广播消息。 + +#### 请求格式 +```json +{ + "id": <整数,请求唯一标识>, + "method": "subscribe", + "params": { + "topic": <二进制,订阅的主题名称,必填> + } +} +``` + +#### 响应格式 +- 成功响应: + ```json + { + "id": <与请求id一致>, + "result": "ok" + } + ``` +- 失败响应: + ```json + { + "id": <与请求id一致>, + "error": { + "code": -1, + "message": "错误描述" + } + } + ``` + +#### 处理逻辑 +通过 `efka_subscription:subscribe(Topic, self())` 完成订阅,订阅成功后客户端会收到该主题的广播消息。 + + +### 3. 指标数据上报(metric_data) +#### 功能 +向服务进程上报设备指标数据。 + +#### 请求格式 +```json +{ + "method": "metric_data", + "params": { + "route_key": <二进制,路由键,必填>, + "metric": <指标数据,必填> + } +} +``` + +#### 响应处理 +服务端接收后无返回消息(处理逻辑:`efka_service:metric_data(ServicePid, DeviceUUID, RouteKey, Metric)`) + +## 五、基础交互协议 +1. **Ping/Pong 心跳**: + - 客户端发送 `ping` 消息 + - 服务端回复 `pong` 消息,维持连接 + +2. **未知消息处理**: + - 客户端发送未定义格式的消息时,服务端记录错误并关闭连接 + +## 七、典型交互流程 +1. 客户端发起 WebSocket 连接 +2. 客户端发送 `register` 请求完成注册 +3. 客户端发送 `subscribe` 请求订阅目标主题 +4. 客户端通过 `metric_data` 上报指标数据 / 通过 `event` 发送事件 +5. 服务端向客户端推送已订阅主题的消息(`publish` 方法) +6. 连接关闭(主动断开或异常终止) \ No newline at end of file diff --git a/apps/docker/src/docker.app.src b/apps/docker/src/docker.app.src new file mode 100644 index 0000000..2ac7ed5 --- /dev/null +++ b/apps/docker/src/docker.app.src @@ -0,0 +1,18 @@ +{application, docker, + [{description, "Docker integration application"}, + {vsn, "0.1.0"}, + {registered, []}, + {mod, {docker_app, []}}, + {applications, + [ + gun, + kernel, + stdlib + ]}, + {env, [ + {root_dir, "/usr/local/code/efka"} + ]}, + {modules, []}, + {licenses, ["Apache-2.0"]}, + {links, []} + ]}. diff --git a/apps/docker/src/docker_app.erl b/apps/docker/src/docker_app.erl new file mode 100644 index 0000000..5975e71 --- /dev/null +++ b/apps/docker/src/docker_app.erl @@ -0,0 +1,18 @@ +%%%------------------------------------------------------------------- +%% @doc Docker application entry point. +%% @end +%%%------------------------------------------------------------------- + +-module(docker_app). + +-behaviour(application). + +-export([start/2, stop/1]). + +-spec start(term(), term()) -> {ok, pid()} | {error, term()}. +start(_StartType, _StartArgs) -> + docker_sup:start_link(). + +-spec stop(term()) -> ok. +stop(_State) -> + ok. diff --git a/apps/docker/src/docker_client.erl b/apps/docker/src/docker_client.erl new file mode 100644 index 0000000..ff67ef4 --- /dev/null +++ b/apps/docker/src/docker_client.erl @@ -0,0 +1,292 @@ +%%%------------------------------------------------------------------- +%%% @doc +%%% Short-lived Docker API client process. +%%% +%%% Each request owns one process and one gun Unix socket connection. +%%% The process exits after the Docker API response finishes. +%%% @end +%%%------------------------------------------------------------------- +-module(docker_client). + +%% API +-export([request/4, start_stream/5, start_stream_no_timeout/5]). + +-define(DOCKER_SOCKET, "/var/run/docker.sock"). +-define(RESPONSE_TIMEOUT, 5000). +-define(BODY_TIMEOUT, 10000). +-define(STREAM_BODY_TIMEOUT, 30000). +-define(CLIENT_TIMEOUT, 60000). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec request(Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) -> + {ok, StatusCode :: integer(), RespHeaders :: proplists:proplist(), RespBody :: binary()} | {error, any()}. +request(Method, Path, Body, Headers) when is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) -> + Owner = self(), + Ref = make_ref(), + {Pid, MRef} = spawn_monitor(fun() -> request_worker(Owner, Ref, Method, Path, Body, Headers) end), + receive + {docker_client, Ref, {result, Result}} -> + erlang:demonitor(MRef, [flush]), + Result; + {'DOWN', MRef, process, Pid, Reason} -> + {error, {client_down, Reason}} + after ?CLIENT_TIMEOUT -> + exit(Pid, shutdown), + erlang:demonitor(MRef, [flush]), + {error, timeout} + end. + +-spec start_stream(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) -> + {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}. +start_stream(Owner, Method, Path, Body, Headers) + when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) -> + Ref = make_ref(), + {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, ?RESPONSE_TIMEOUT, ?STREAM_BODY_TIMEOUT) end), + {ok, Ref, Pid, MRef}. + +-spec start_stream_no_timeout(Owner :: pid(), Method :: string(), Path :: string(), Body :: binary(), Headers :: list()) -> + {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()}. +start_stream_no_timeout(Owner, Method, Path, Body, Headers) + when is_pid(Owner), is_list(Method), is_list(Path), is_binary(Body), is_list(Headers) -> + Ref = make_ref(), + {Pid, MRef} = spawn_monitor(fun() -> stream_worker(Owner, Ref, Method, Path, Body, Headers, infinity, infinity) end), + {ok, Ref, Pid, MRef}. + +%%%=================================================================== +%%% Worker functions +%%%=================================================================== + +-spec request_worker(pid(), reference(), string(), string(), binary(), list()) -> ok. +request_worker(Owner, Ref, Method, Path, Body, Headers) -> + Result = do_request(Method, Path, Body, Headers), + send_owner(Owner, Ref, {result, Result}). + +-spec stream_worker(pid(), reference(), string(), string(), binary(), list(), timeout(), timeout()) -> ok. +stream_worker(Owner, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) -> + OwnerRef = erlang:monitor(process, Owner), + _Result = do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout), + erlang:demonitor(OwnerRef, [flush]), + ok. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== + +-spec do_request(string(), string(), binary(), list()) -> + {ok, integer(), proplists:proplist(), binary()} | {error, any()}. +do_request(Method, Path, Body, Headers) -> + case open_connection() of + {ok, ConnPid} -> + try + StreamRef = gun:request(ConnPid, Method, Path, Headers, Body), + receive_response(ConnPid, StreamRef) + after + close_connection(ConnPid) + end; + {error, Reason} -> + {error, Reason} + end. + +-spec do_stream(pid(), reference(), reference(), string(), string(), binary(), list(), timeout(), timeout()) -> + ok | {error, binary()}. +do_stream(Owner, OwnerRef, Ref, Method, Path, Body, Headers, ResponseTimeout, BodyTimeout) -> + case open_connection() of + {ok, ConnPid} -> + try + StreamRef = gun:request(ConnPid, Method, Path, Headers, Body), + receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout) + after + close_connection(ConnPid) + end; + {error, Reason0} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason} + end. + +-spec receive_response(pid(), reference()) -> + {ok, integer(), proplists:proplist(), binary()} | {error, any()}. +receive_response(ConnPid, StreamRef) -> + receive + {gun_response, ConnPid, StreamRef, nofin, Status, Headers} -> + receive_body(ConnPid, StreamRef, Status, Headers, <<>>); + {gun_response, ConnPid, StreamRef, fin, Status, Headers} -> + {ok, Status, Headers, <<>>}; + {gun_error, ConnPid, StreamRef, Reason} -> + {error, {http_error, Reason}}; + {gun_error, ConnPid, Reason} -> + {error, {http_error, Reason}}; + {gun_down, ConnPid, _, Reason, _} -> + {error, {http_closed, Reason}} + after ?RESPONSE_TIMEOUT -> + {error, timeout} + end. + +-spec receive_body(pid(), reference(), integer(), proplists:proplist(), iodata()) -> + {ok, integer(), proplists:proplist(), binary()} | {error, any()}. +receive_body(ConnPid, StreamRef, Status, Headers, Acc) -> + receive + {gun_data, ConnPid, StreamRef, fin, Data} -> + Body = iolist_to_binary([Acc, Data]), + {ok, Status, Headers, Body}; + {gun_data, ConnPid, StreamRef, nofin, Data} -> + receive_body(ConnPid, StreamRef, Status, Headers, [Acc, Data]); + {gun_error, ConnPid, StreamRef, Reason} -> + {error, {http_error, Reason}}; + {gun_error, ConnPid, Reason} -> + {error, {http_error, Reason}}; + {gun_down, ConnPid, _, Reason, _} -> + {error, {http_closed, Reason}} + after ?BODY_TIMEOUT -> + {error, timeout} + end. + +-spec receive_stream_response(pid(), reference(), reference(), pid(), reference(), timeout(), timeout()) -> ok | {error, binary()}. +receive_stream_response(Owner, OwnerRef, Ref, ConnPid, StreamRef, ResponseTimeout, BodyTimeout) -> + receive + {gun_response, ConnPid, StreamRef, nofin, Status, Headers} when Status >= 200, Status < 300 -> + send_owner(Owner, Ref, {response, Status, Headers}), + receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout); + {gun_response, ConnPid, StreamRef, fin, Status, Headers} when Status >= 200, Status < 300 -> + send_owner(Owner, Ref, {response, Status, Headers}), + send_owner(Owner, Ref, done), + ok; + {gun_response, ConnPid, StreamRef, nofin, Status, _Headers} -> + receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, <<>>, stream_error_body_timeout(BodyTimeout)); + {gun_response, ConnPid, StreamRef, fin, Status, _Headers} -> + Reason = http_status_error(Status, <<>>), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_error, ConnPid, StreamRef, Reason0} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_error, ConnPid, Reason0} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_down, ConnPid, _, Reason0, _} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {'DOWN', OwnerRef, process, _Pid, Reason0} -> + {error, format_error({owner_down, Reason0})} + after ResponseTimeout -> + send_owner(Owner, Ref, {error, <<"处理超时"/utf8>>}), + {error, <<"timeout">>} + end. + +-spec receive_stream_body(pid(), reference(), reference(), pid(), reference(), timeout()) -> ok | {error, binary()}. +receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout) -> + receive + {gun_data, ConnPid, StreamRef, fin, Data} -> + maybe_send_stream_data(Owner, Ref, Data), + send_owner(Owner, Ref, done), + ok; + {gun_data, ConnPid, StreamRef, nofin, Data} -> + maybe_send_stream_data(Owner, Ref, Data), + receive_stream_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, BodyTimeout); + {gun_error, ConnPid, StreamRef, Reason0} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_error, ConnPid, Reason0} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_down, ConnPid, _, Reason0, _} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {'DOWN', OwnerRef, process, _Pid, Reason0} -> + {error, format_error({owner_down, Reason0})} + after BodyTimeout -> + send_owner(Owner, Ref, {error, <<"timeout">>}), + {error, <<"timeout">>} + end. + +-spec receive_stream_error_body(pid(), reference(), reference(), pid(), reference(), integer(), iodata(), timeout()) -> + {error, binary()}. +receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, Acc, BodyTimeout) -> + receive + {gun_data, ConnPid, StreamRef, fin, Data} -> + Reason = http_status_error(Status, iolist_to_binary([Acc, Data])), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_data, ConnPid, StreamRef, nofin, Data} -> + receive_stream_error_body(Owner, OwnerRef, Ref, ConnPid, StreamRef, Status, [Acc, Data], BodyTimeout); + {gun_error, ConnPid, StreamRef, Reason0} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_error, ConnPid, Reason0} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {gun_down, ConnPid, _, Reason0, _} -> + Reason = format_error(Reason0), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason}; + {'DOWN', OwnerRef, process, _Pid, Reason0} -> + {error, format_error({owner_down, Reason0})} + after BodyTimeout -> + Reason = http_status_error(Status, iolist_to_binary(Acc)), + send_owner(Owner, Ref, {error, Reason}), + {error, Reason} + end. + +-spec stream_error_body_timeout(timeout()) -> timeout(). +stream_error_body_timeout(infinity) -> + infinity; +stream_error_body_timeout(_BodyTimeout) -> + ?BODY_TIMEOUT. + +-spec open_connection() -> {ok, pid()} | {error, any()}. +open_connection() -> + case gun:open_unix(?DOCKER_SOCKET, #{}) of + {ok, ConnPid} -> + case gun:await_up(ConnPid) of + {ok, _} -> + {ok, ConnPid}; + {error, Reason} -> + close_connection(ConnPid), + {error, Reason} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec close_connection(pid()) -> ok. +close_connection(ConnPid) -> + catch gun:close(ConnPid), + ok. + +-spec send_owner(pid(), reference(), term()) -> ok. +send_owner(Owner, Ref, Event) -> + Owner ! {docker_client, Ref, Event}, + ok. + +-spec maybe_send_stream_data(pid(), reference(), iodata()) -> ok. +maybe_send_stream_data(Owner, Ref, Data) -> + case iolist_to_binary(Data) of + <<>> -> + ok; + DataBin -> + send_owner(Owner, Ref, {data, DataBin}) + end. + +-spec http_status_error(integer(), binary()) -> binary(). +http_status_error(Status, <<>>) -> + iolist_to_binary(io_lib:format("docker http status ~B", [Status])); +http_status_error(Status, Body) when is_binary(Body) -> + StatusBin = integer_to_binary(Status), + <<"docker http status ", StatusBin/binary, ": ", Body/binary>>. + +-spec format_error(term()) -> binary(). +format_error(Reason) when is_binary(Reason) -> + Reason; +format_error(Reason) -> + iolist_to_binary(io_lib:format("~p", [Reason])). diff --git a/apps/docker/src/docker_commands.erl b/apps/docker/src/docker_commands.erl new file mode 100644 index 0000000..ec7d4eb --- /dev/null +++ b/apps/docker/src/docker_commands.erl @@ -0,0 +1,295 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2025, +%%% @doc +%%% +%%% @end +%%% Created : 15. 9月 2025 16:11 +%%%------------------------------------------------------------------- +-module(docker_commands). +-author("anlicheng"). + +%% API +-export([pull_image/1, check_image_exist/1]). +-export([create_container/2, check_container_exist/1, is_container_running/1, + start_container/1, stop_container/1, stop_container/2, remove_container/1, remove_container/3, kill_container/1, kill_container/2, + get_containers/0]). + +-spec pull_image(Image :: binary()) -> {ok, Ref :: reference(), Pid :: pid(), MRef :: reference()} | {error, Reason :: any()}. +pull_image(Image) when is_binary(Image) -> + Url = lists:flatten(io_lib:format("/images/create?fromImage=~s", [binary_to_list(Image)])), + docker_client:start_stream_no_timeout(self(), "POST", Url, <<>>, []). + +-spec check_image_exist(Image :: binary()) -> boolean(). +check_image_exist(Image) when is_binary(Image) -> + EncodedImage = uri_string:quote(Image), + Url = lists:flatten(io_lib:format("/images/~s/json", [binary_to_list(EncodedImage)])), + case docker_client:request("GET", Url, <<"">>, []) of + {ok, 200, _Headers, _Resp} -> + true; + {ok, 404, _, _} -> + false; + _ -> + false + end. + +-spec create_container(ContainerDir :: string(), Params :: map()) -> + {ok, ContainerId :: binary()} | {error, Reason :: any()}. +create_container(ContainerDir, #{<<"container_name">> := ContainerName0, <<"create">> := CreateOpts}) + when is_list(ContainerDir), is_binary(ContainerName0), is_map(CreateOpts) -> + Url = lists:flatten(io_lib:format("/containers/create?name=~s", [binary_to_list(ContainerName0)])), + Options = docker_container_builder:build_options(ContainerName0, ContainerDir, CreateOpts), + display_options(Options), + + Body = iolist_to_binary(json:encode(Options)), + + Headers = [ + {<<"Content-Type">>, <<"application/json">>} + ], + + case docker_client:request("POST", Url, Body, Headers) of + {ok, 201, _Headers, Resp} -> + case catch json:decode(Resp) of + #{<<"Id">> := ContainerId} when is_binary(ContainerId) -> + {ok, ContainerId}; + _ -> + {error, Resp} + end; + {ok, _StatusCode, _, ErrorResp} -> + case catch json:decode(ErrorResp) of + #{<<"message">> := Msg} -> + {error, Msg}; + _ -> + {error, ErrorResp} + end; + {error, Reason} -> + {error, Reason} + end; +create_container(_ContainerDir, _Params) -> + {error, <<"invalid container params">>}. + +-spec is_container_running(ContainerId :: binary()) -> boolean(). +is_container_running(ContainerId) when is_binary(ContainerId) -> + case inspect_container(ContainerId) of + {ok, #{<<"State">> := #{<<"Running">> := Running}}} -> + Running; + {error, _} -> + false + end. + +-spec check_container_exist(ContainerName :: binary()) -> boolean(). +check_container_exist(ContainerName) when is_binary(ContainerName) -> + case inspect_container(ContainerName) of + {ok, #{<<"Id">> := Id}} when is_binary(Id) -> + true; + _ -> + false + end. + +-spec start_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}. +start_container(ContainerName) when is_binary(ContainerName) -> + Url = lists:flatten(io_lib:format("/containers/~s/start", [binary_to_list(ContainerName)])), + Headers = [ + {<<"Content-Type">>, <<"application/json">>} + ], + case docker_client:request("POST", Url, <<>>, Headers) of + {ok, 204, _Headers, _} -> + ok; + {ok, 304, _Headers, _} -> + {error, <<"container already started">>}; + {ok, _StatusCode, _Header, ErrorResp} -> + case catch json:decode(ErrorResp) of + #{<<"message">> := Msg} -> + {error, Msg}; + _ -> + {error, ErrorResp} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec stop_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}. +stop_container(ContainerName) when is_binary(ContainerName) -> + stop_container(ContainerName, 0). + +-spec stop_container(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> ok | {error, Reason :: binary()}. +stop_container(ContainerName, TimeoutSeconds) when is_binary(ContainerName), is_integer(TimeoutSeconds), TimeoutSeconds >= 0 -> + Url = build_stop_container_url(ContainerName, TimeoutSeconds), + Headers = [ + {<<"Content-Type">>, <<"application/json">>} + ], + case docker_client:request("POST", Url, <<>>, Headers) of + {ok, 204, _Headers, _} -> + ok; + {ok, 304, _Headers, _} -> + {error, <<"container already stopped">>}; + {ok, _StatusCode, _Header, ErrorResp} -> + case catch json:decode(ErrorResp) of + #{<<"message">> := Msg} -> + {error, Msg}; + _ -> + {error, ErrorResp} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec kill_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}. +kill_container(ContainerName) when is_binary(ContainerName) -> + kill_container(ContainerName, <<>>). + +-spec kill_container(ContainerName :: binary(), Signal :: binary()) -> ok | {error, Reason :: binary()}. +kill_container(ContainerName, Signal) when is_binary(ContainerName), is_binary(Signal) -> + Url = build_kill_container_url(ContainerName, Signal), + Headers = [ + {<<"Content-Type">>, <<"application/json">>} + ], + case docker_client:request("POST", Url, <<>>, Headers) of + {ok, 204, _Headers, _} -> + ok; + {ok, _StatusCode, _Header, ErrorResp} -> + case catch json:decode(ErrorResp) of + #{<<"message">> := Msg} -> + {error, Msg}; + _ -> + {error, ErrorResp} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec remove_container(ContainerName :: binary()) -> ok | {error, Reason :: binary()}. +remove_container(ContainerName) when is_binary(ContainerName) -> + remove_container(ContainerName, false, false). + +-spec remove_container(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> ok | {error, Reason :: binary()}. +remove_container(ContainerName, Force, RemoveVolumes) + when is_binary(ContainerName), is_boolean(Force), is_boolean(RemoveVolumes) -> + Url = build_remove_container_url(ContainerName, Force, RemoveVolumes), + Headers = [ + {<<"Content-Type">>, <<"application/json">>} + ], + case docker_client:request("DELETE", Url, <<>>, Headers) of + {ok, 204, _Headers, _} -> + ok; + {ok, 304, _Headers, _} -> + {error, <<"container already stopped">>}; + {ok, _StatusCode, _Header, ErrorResp} -> + case catch json:decode(ErrorResp) of + #{<<"message">> := Msg} -> + {error, Msg}; + _ -> + {error, ErrorResp} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec get_containers() -> {ok, Containers :: [map()]} | {error, Reason :: binary()}. +get_containers() -> + Url = "/containers/json?all=true", + Headers = [ + {<<"Content-Type">>, <<"application/json">>} + ], + case docker_client:request("GET", Url, <<>>, Headers) of + {ok, 200, _Headers, ContainersBin} -> + Containers = json:decode(ContainersBin), + {ok, Containers}; + {ok, _StatusCode, _Header, ErrorResp} -> + case catch json:decode(ErrorResp) of + #{<<"message">> := Msg} -> + {error, Msg}; + _ -> + {error, ErrorResp} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec inspect_container(ContainerId :: binary()) -> {ok, Json :: map()} | {error, Error :: any()}. +inspect_container(ContainerId) when is_binary(ContainerId) -> + Url = lists:flatten(io_lib:format("/containers/~s/json", [binary_to_list(ContainerId)])), + Headers = [ + {<<"Content-Type">>, <<"application/json">>} + ], + case docker_client:request("GET", Url, <<>>, Headers) of + {ok, 200, _Headers, Resp} -> + decode_container_inspect_summary(Resp); + {ok, _StatusCode, _Header, ErrorResp} -> + case catch json:decode(ErrorResp) of + #{<<"message">> := Msg} -> + {error, Msg}; + _ -> + {error, ErrorResp} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec decode_container_inspect_summary(binary()) -> {ok, map()} | {error, binary()}. +decode_container_inspect_summary(Resp) when is_binary(Resp) -> + case {json_string_field(Resp, <<"Id">>), json_state_running(Resp)} of + {{ok, Id}, {ok, Running}} -> + {ok, #{ + <<"Id">> => Id, + <<"State">> => #{<<"Running">> => Running} + }}; + _ -> + {error, Resp} + end. + +-spec json_string_field(binary(), binary()) -> {ok, binary()} | error. +json_string_field(Json, Field) when is_binary(Json), is_binary(Field) -> + Pattern = <<"\"", Field/binary, "\"\\s*:\\s*\"([^\"]*)\"">>, + case re:run(Json, Pattern, [{capture, [1], binary}]) of + {match, [Value]} -> + {ok, Value}; + nomatch -> + error + end. + +-spec json_state_running(binary()) -> {ok, boolean()} | error. +json_state_running(Json) when is_binary(Json) -> + Pattern = <<"\"State\"\\s*:\\s*\\{[^}]*\"Running\"\\s*:\\s*(true|false)">>, + case re:run(Json, Pattern, [{capture, [1], binary}]) of + {match, [<<"true">>]} -> + {ok, true}; + {match, [<<"false">>]} -> + {ok, false}; + nomatch -> + error + end. + +-spec display_options(Options :: map()) -> ok. +display_options(Options) when is_map(Options) -> + logger:debug("deploy options: ~p", [iolist_to_binary(json:encode(Options))]), + %lists:foreach(fun({K, V}) -> logger:debug("~p => ~p", [K, V]) end, maps:to_list(Options)), + ok. + +-spec build_stop_container_url(ContainerName :: binary(), TimeoutSeconds :: non_neg_integer()) -> string(). +build_stop_container_url(ContainerName, 0) -> + lists:flatten(io_lib:format("/containers/~s/stop", [binary_to_list(ContainerName)])); +build_stop_container_url(ContainerName, TimeoutSeconds) -> + lists:flatten(io_lib:format("/containers/~s/stop?t=~B", [binary_to_list(ContainerName), TimeoutSeconds])). + +-spec build_kill_container_url(ContainerName :: binary(), Signal :: binary()) -> string(). +build_kill_container_url(ContainerName, <<>>) -> + lists:flatten(io_lib:format("/containers/~s/kill", [binary_to_list(ContainerName)])); +build_kill_container_url(ContainerName, Signal) -> + lists:flatten(io_lib:format("/containers/~s/kill?signal=~s", [binary_to_list(ContainerName), binary_to_list(Signal)])). + +-spec build_remove_container_url(ContainerName :: binary(), Force :: boolean(), RemoveVolumes :: boolean()) -> string(). +build_remove_container_url(ContainerName, Force, RemoveVolumes) -> + ForceValue = boolean_to_query_value(Force), + RemoveVolumesValue = boolean_to_query_value(RemoveVolumes), + lists:flatten(io_lib:format("/containers/~s?force=~s&v=~s", [ + binary_to_list(ContainerName), + ForceValue, + RemoveVolumesValue + ])). + +-spec boolean_to_query_value(boolean()) -> string(). +boolean_to_query_value(true) -> + "true"; +boolean_to_query_value(false) -> + "false". diff --git a/apps/docker/src/docker_commands_tests.erl b/apps/docker/src/docker_commands_tests.erl new file mode 100644 index 0000000..043c27d --- /dev/null +++ b/apps/docker/src/docker_commands_tests.erl @@ -0,0 +1,321 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2025, +%%% @doc +%%% +%%% @end +%%% Created : 23. 9月 2025 17:23 +%%%------------------------------------------------------------------- +-module(docker_commands_tests). +-author("anlicheng"). + +%% API +-export([ + test_all/0, + test_pull/0, + test_check_image_exist/0, + test_check_image_not_exist/0, + test_create_container/0, + test_create_container_without_create_options/0, + test_create_container_patches_options/0, + test_check_container_exist/0, + test_check_container_not_exist/0, + test_start_container/0, + test_stop_container/0, + test_stop_container_with_timeout/0, + test_kill_container/0, + test_kill_container_with_signal/0, + test_remove_container/0, + test_remove_container_with_options/0, + test_get_containers/0 +]). + +-define(TEST_IMAGE, <<"docker.1ms.run/library/nginx:latest">>). +-define(TEST_CMD, [<<"nginx">>, <<"-g">>, <<"daemon off;">>]). + +-spec test_all() -> ok. +test_all() -> + ok = test_pull(), + ok = test_check_image_exist(), + ok = test_check_image_not_exist(), + ok = test_create_container(), + ok = test_create_container_without_create_options(), + ok = test_create_container_patches_options(), + ok = test_check_container_exist(), + ok = test_check_container_not_exist(), + ok = test_start_container(), + ok = test_stop_container(), + ok = test_stop_container_with_timeout(), + ok = test_kill_container(), + ok = test_kill_container_with_signal(), + ok = test_remove_container(), + ok = test_remove_container_with_options(), + ok = test_get_containers(), + ok. + +-spec test_pull() -> ok. +test_pull() -> + {ok, Ref, Pid, MRef} = docker_commands:pull_image(?TEST_IMAGE), + await_pull(Ref, Pid, MRef). + +-spec await_pull(reference(), pid(), reference()) -> ok. +await_pull(Ref, Pid, MRef) -> + receive + {docker_client, Ref, {response, _Status, _Headers}} -> + await_pull(Ref, Pid, MRef); + {docker_client, Ref, {data, Data}} -> + logger:debug("msg is: ~p", [Data]), + await_pull(Ref, Pid, MRef); + {docker_client, Ref, done} -> + erlang:demonitor(MRef, [flush]), + ok; + {docker_client, Ref, {error, Reason}} -> + erlang:demonitor(MRef, [flush]), + error({pull_failed, Reason}); + {'DOWN', MRef, process, Pid, Reason} -> + error({pull_client_down, Reason}) + end. + +-spec test_check_image_exist() -> ok. +test_check_image_exist() -> + true = docker_commands:check_image_exist(?TEST_IMAGE), + ok. + +-spec test_check_image_not_exist() -> ok. +test_check_image_not_exist() -> + false = docker_commands:check_image_exist(<<"docker.1ms.run/library/not-exists-for-efka-tests:latest">>), + ok. + +-spec test_create_container() -> ok. +test_create_container() -> + Name = test_container_name(<<"create">>), + ContainerDir = prepare_container_dir(Name), + {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)), + true = is_binary(ContainerId), + ok. + +-spec test_create_container_without_create_options() -> ok. +test_create_container_without_create_options() -> + Name = test_container_name(<<"create-default">>), + ContainerDir = prepare_container_dir(Name), + Options = docker_container_builder:build_options(Name, ContainerDir, undefined), + assert_patched_default_options(Name, ContainerDir, Options), + ok. + +-spec test_create_container_patches_options() -> ok. +test_create_container_patches_options() -> + Name = test_container_name(<<"create-patch">>), + ContainerDir = prepare_container_dir(Name), + Create = #{ + config => #{ + image => ?TEST_IMAGE, + cmd => ?TEST_CMD, + env => [<<"EXISTING_ENV=1">>], + volumes => [<<"/data">>] + }, + host_config => #{ + binds => [<<"/tmp:/tmp">>] + } + }, + Params = #{ + container_name => Name, + create => Create + }, + Options = docker_container_builder:build_options(Name, ContainerDir, Create), + assert_patched_default_options(Name, ContainerDir, Options), + assert_existing_options_preserved(Options), + try + ok = test_pull(), + {ok, _ContainerId} = docker_commands:create_container(ContainerDir, Params), + ok + after + ok + %cleanup_container(Name) + end. + +-spec test_check_container_exist() -> ok. +test_check_container_exist() -> + Name = test_container_name(<<"exist">>), + with_created_container(Name, fun(_ContainerDir, _ContainerId) -> + true = docker_commands:check_container_exist(Name), + ok + end). + +-spec test_check_container_not_exist() -> ok. +test_check_container_not_exist() -> + false = docker_commands:check_container_exist(test_container_name(<<"missing">>)), + ok. + +-spec test_start_container() -> ok. +test_start_container() -> + Name = test_container_name(<<"start">>), + with_created_container(Name, fun(_ContainerDir, ContainerId) -> + ok = docker_commands:start_container(Name), + true = docker_commands:is_container_running(ContainerId), + ok + end). + +-spec test_stop_container() -> ok. +test_stop_container() -> + Name = test_container_name(<<"stop">>), + with_started_container(Name, fun(_ContainerDir, ContainerId) -> + ok = docker_commands:stop_container(Name), + false = docker_commands:is_container_running(ContainerId), + ok + end). + +-spec test_stop_container_with_timeout() -> ok. +test_stop_container_with_timeout() -> + Name = test_container_name(<<"stop-timeout">>), + with_started_container(Name, fun(_ContainerDir, ContainerId) -> + ok = docker_commands:stop_container(Name, 1), + false = docker_commands:is_container_running(ContainerId), + ok + end). + +-spec test_kill_container() -> ok. +test_kill_container() -> + Name = test_container_name(<<"kill">>), + with_started_container(Name, fun(_ContainerDir, ContainerId) -> + ok = docker_commands:kill_container(Name), + timer:sleep(200), + false = docker_commands:is_container_running(ContainerId), + ok + end). + +-spec test_kill_container_with_signal() -> ok. +test_kill_container_with_signal() -> + Name = test_container_name(<<"kill-signal">>), + with_started_container(Name, fun(_ContainerDir, ContainerId) -> + ok = docker_commands:kill_container(Name, <<"SIGKILL">>), + timer:sleep(200), + false = docker_commands:is_container_running(ContainerId), + ok + end). + +-spec test_remove_container() -> ok. +test_remove_container() -> + Name = test_container_name(<<"remove">>), + with_created_container(Name, fun(_ContainerDir, _ContainerId) -> + ok = docker_commands:remove_container(Name), + false = docker_commands:check_container_exist(Name), + ok + end, false). + +-spec test_remove_container_with_options() -> ok. +test_remove_container_with_options() -> + Name = test_container_name(<<"remove-opts">>), + with_started_container(Name, fun(_ContainerDir, _ContainerId) -> + ok = docker_commands:remove_container(Name, true, false), + false = docker_commands:check_container_exist(Name), + ok + end, false). + +-spec test_get_containers() -> ok. +test_get_containers() -> + Name = test_container_name(<<"list">>), + with_created_container(Name, fun(_ContainerDir, ContainerId) -> + {ok, Containers} = docker_commands:get_containers(), + logger:debug("list containers: ~p", [Containers]), + true = is_list(Containers), + true = contains_container(Name, ContainerId, Containers), + ok + end). + +-spec with_created_container(binary(), fun((string(), binary()) -> ok)) -> ok. +with_created_container(Name, Fun) -> + with_created_container(Name, Fun, true). + +-spec with_created_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok. +with_created_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) -> + ContainerDir = prepare_container_dir(Name), + try + ok = test_pull(), + {ok, ContainerId} = docker_commands:create_container(ContainerDir, minimal_params(Name)), + ok = Fun(ContainerDir, ContainerId) + after + case Cleanup of + true -> + cleanup_container(Name); + false -> + ok + end + end. + +-spec with_started_container(binary(), fun((string(), binary()) -> ok)) -> ok. +with_started_container(Name, Fun) -> + with_started_container(Name, Fun, true). + +-spec with_started_container(binary(), fun((string(), binary()) -> ok), boolean()) -> ok. +with_started_container(Name, Fun, Cleanup) when is_binary(Name), is_function(Fun, 2), is_boolean(Cleanup) -> + with_created_container(Name, fun(ContainerDir, ContainerId) -> + ok = docker_commands:start_container(Name), + ok = Fun(ContainerDir, ContainerId) + end, Cleanup). + +-spec minimal_params(binary()) -> map(). +minimal_params(Name) when is_binary(Name) -> + #{ + container_name => Name, + create => #{ + config => #{ + image => ?TEST_IMAGE, + cmd => ?TEST_CMD + } + } + }. + +-spec prepare_container_dir(binary()) -> string(). +prepare_container_dir(Name) when is_binary(Name) -> + Dir = lists:flatten(io_lib:format("/usr/local/code/efka/~ts/", [Name])), + ok = filelib:ensure_dir(Dir ++ "placeholder"), + ok = file:write_file(Dir ++ "service.conf", <<>>, [write]), + Dir. + +-spec cleanup_container(binary()) -> ok. +cleanup_container(Name) when is_binary(Name) -> + _ = docker_commands:remove_container(Name, true, false), + ok. + +-spec assert_patched_default_options(binary(), string(), map()) -> ok. +assert_patched_default_options(Name, ContainerDir, Options) + when is_binary(Name), is_list(ContainerDir), is_map(Options) -> + ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)), + ExpectedBind = <>, + #{<<"Env">> := Env, + <<"Volumes">> := Volumes, + <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options, + true = lists:member(<<"CONTAINER_NAME=", Name/binary>>, Env), + true = maps:is_key(<<"/usr/local/etc/service.conf">>, Volumes), + true = lists:member(ExpectedBind, Binds), + ok. + +-spec assert_existing_options_preserved(map()) -> ok. +assert_existing_options_preserved(Options) when is_map(Options) -> + #{<<"Env">> := Env, + <<"Volumes">> := Volumes, + <<"HostConfig">> := #{<<"Binds">> := Binds}} = Options, + true = lists:member(<<"EXISTING_ENV=1">>, Env), + true = maps:is_key(<<"/data">>, Volumes), + true = lists:member(<<"/tmp:/tmp">>, Binds), + ok. + +-spec contains_container(binary(), binary(), [map()]) -> boolean(). +contains_container(Name, ContainerId, Containers) when is_binary(Name), is_binary(ContainerId), is_list(Containers) -> + lists:any(fun(Container) -> container_matches(Name, ContainerId, Container) end, Containers). + +-spec container_matches(binary(), binary(), map()) -> boolean(). +container_matches(Name, ContainerId, #{<<"Id">> := Id, <<"Names">> := Names}) when is_binary(Id), is_list(Names) -> + lists:member(<<"/", Name/binary>>, Names) orelse has_id_prefix(ContainerId, Id); +container_matches(_Name, _ContainerId, _Container) -> + false. + +-spec has_id_prefix(binary(), binary()) -> boolean(). +has_id_prefix(ExpectedId, ActualId) when is_binary(ExpectedId), is_binary(ActualId) -> + PrefixLen = erlang:min(byte_size(ExpectedId), byte_size(ActualId)), + binary:part(ExpectedId, 0, PrefixLen) =:= binary:part(ActualId, 0, PrefixLen). + +-spec test_container_name(binary()) -> binary(). +test_container_name(Prefix) when is_binary(Prefix) -> + Suffix = integer_to_binary(erlang:unique_integer([positive])), + <<"efka-test-", Prefix/binary, "-", Suffix/binary>>. diff --git a/apps/docker/src/docker_container_builder.erl b/apps/docker/src/docker_container_builder.erl new file mode 100644 index 0000000..f14e089 --- /dev/null +++ b/apps/docker/src/docker_container_builder.erl @@ -0,0 +1,379 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2025, +%%% @doc +%%% +%%% @end +%%% Created : 21. 4月 2026 15:58 +%%%------------------------------------------------------------------- +-module(docker_container_builder). +-author("anlicheng"). + +%% API +-export([build_options/3]). + +-spec build_options(binary(), string(), map() | undefined) -> map(). +build_options(ContainerName, ContainerDir, Create0) when is_binary(ContainerName), is_list(ContainerDir) -> + ConfigFile = list_to_binary(docker_helper:get_config_file(ContainerDir)), + Create = patch_create_options(ContainerName, ConfigFile, Create0), + build_create_options(Create). + +-spec patch_create_options(binary(), binary(), map() | undefined) -> map(). +patch_create_options(ContainerName, ConfigFile, undefined) -> + patch_create_options(ContainerName, ConfigFile, #{}); +patch_create_options(ContainerName, ConfigFile, Create0) + when is_binary(ContainerName), is_binary(ConfigFile), is_map(Create0) -> + Config = patch_container_config(ContainerName, ensure_container_config(field(Create0, <<"config">>, undefined))), + HostConfig = patch_host_config(ConfigFile, ensure_host_config(field(Create0, <<"host_config">>, undefined))), + Create0#{ + <<"config">> => Config, + <<"host_config">> => HostConfig + }. + +-spec patch_container_config(binary(), map()) -> map(). +patch_container_config(ContainerName, Config0) when is_binary(ContainerName), is_map(Config0) -> + Env0 = [to_binary(EnvItem) || EnvItem <- field(Config0, <<"env">>, [])], + Volumes0 = [to_binary(Volume) || Volume <- field(Config0, <<"volumes">>, [])], + ConfigVolume = <<"/usr/local/etc/service.conf">>, + Envs = add_unique_front([<<"CONTAINER_NAME=", ContainerName/binary>>], Env0), + Volumes = add_unique_front([ConfigVolume], Volumes0), + Config0#{ + <<"env">> => Envs, + <<"volumes">> => Volumes + }. + +-spec patch_host_config(binary(), map()) -> map(). +patch_host_config(ConfigFile, HostConfig0) when is_binary(ConfigFile), is_map(HostConfig0) -> + Binds0 = [to_binary(Bind) || Bind <- field(HostConfig0, <<"binds">>, [])], + ConfigBind = <>, + HostConfig0#{<<"binds">> => add_unique_front([ConfigBind], Binds0)}. + +-spec ensure_container_config(map() | undefined) -> map(). +ensure_container_config(undefined) -> + #{}; +ensure_container_config(Config) when is_map(Config) -> + Config. + +-spec ensure_host_config(map() | undefined) -> map(). +ensure_host_config(undefined) -> + #{}; +ensure_host_config(HostConfig) when is_map(HostConfig) -> + HostConfig. + +-spec build_create_options(map()) -> map(). +build_create_options(Create0) when is_map(Create0) -> + Config = field(Create0, <<"config">>, #{}), + HostConfig = field(Create0, <<"host_config">>, #{}), + Endpoints = networking_config_endpoints(field(Create0, <<"networking_config">>, undefined)), + #{ + <<"Image">> => to_binary(field(Config, <<"image">>, <<>>)), + <<"Cmd">> => [to_binary(CommandItem) || CommandItem <- field(Config, <<"cmd">>, [])], + <<"Entrypoint">> => [to_binary(EntrypointItem) || EntrypointItem <- field(Config, <<"entrypoint">>, [])], + <<"Env">> => [to_binary(EnvItem) || EnvItem <- field(Config, <<"env">>, [])], + <<"Labels">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(field(Config, <<"labels">>, #{}))]), + <<"Volumes">> => build_volumes(field(Config, <<"volumes">>, [])), + <<"User">> => to_binary(field(Config, <<"user">>, <<>>)), + <<"WorkingDir">> => to_binary(field(Config, <<"working_dir">>, <<>>)), + <<"Hostname">> => to_binary(field(Config, <<"hostname">>, <<>>)), + <<"ExposedPorts">> => build_expose(field(Config, <<"exposed_ports">>, [])), + <<"NetworkingConfig">> => build_networking_config(Endpoints), + <<"Healthcheck">> => build_healthcheck(field(Config, <<"healthcheck">>, undefined)), + <<"HostConfig">> => fold_merge([ + build_binds(field(HostConfig, <<"binds">>, [])), + build_network_mode(field(HostConfig, <<"network_mode">>, <<>>)), + build_restart(field(HostConfig, <<"restart_policy">>, undefined)), + build_privileged(field(HostConfig, <<"privileged">>, false)), + build_cap_add_drop(field(HostConfig, <<"cap_add">>, []), field(HostConfig, <<"cap_drop">>, [])), + build_devices(field(HostConfig, <<"devices">>, [])), + build_resources( + field(HostConfig, <<"memory">>, 0), + field(HostConfig, <<"memory_reservation">>, 0), + field(HostConfig, <<"nano_cpus">>, 0), + field(HostConfig, <<"cpu_shares">>, 0) + ), + build_port_bindings(field(HostConfig, <<"port_bindings">>, [])), + build_ulimits(field(HostConfig, <<"ulimits">>, [])), + build_tmpfs(field(HostConfig, <<"tmpfs">>, #{})), + build_sysctls(field(HostConfig, <<"sysctls">>, #{})), + build_extra_hosts(field(HostConfig, <<"extra_hosts">>, [])) + ]) + }. + +-spec networking_config_endpoints(map() | undefined) -> [map()]. +networking_config_endpoints(undefined) -> + []; +networking_config_endpoints(NetworkingConfig) when is_map(NetworkingConfig) -> + field(NetworkingConfig, <<"endpoints">>, []). + +-spec fold_merge([map()]) -> map(). +fold_merge(List) -> + lists:foldl(fun maps:merge/2, #{}, List). + +-spec build_expose([map()]) -> map(). +build_expose(Ports) when is_list(Ports) -> + case Ports of + [] -> + #{}; + _ -> + maps:from_list([{normalize_expose_port(Port), #{}} || Port <- Ports]) + end. + +-spec build_volumes([binary() | list() | atom()]) -> map(). +build_volumes(Volumes) when is_list(Volumes) -> + case Volumes of + [] -> + #{}; + _ -> + maps:from_list([{to_binary(ContainerPath), #{}} || ContainerPath <- Volumes]) + end. + +-spec build_binds([binary() | list() | atom()]) -> map(). +build_binds(Binds) when is_list(Binds) -> + case Binds of + [] -> + #{}; + _ -> + #{<<"Binds">> => [to_binary(Bind) || Bind <- Binds]} + end. + +-spec build_networking_config([map()]) -> map(). +build_networking_config(Endpoints) when is_list(Endpoints) -> + case Endpoints of + [] -> + #{}; + _ -> + NetCfg = maps:from_list([{to_binary(Name), #{}} || #{<<"name">> := Name} <- Endpoints]), + #{<<"EndpointsConfig">> => NetCfg} + end. + +-spec build_network_mode(binary() | list() | atom()) -> map(). +build_network_mode(<<>>) -> + #{}; +build_network_mode(NetworkMode) -> + #{<<"NetworkMode">> => to_binary(NetworkMode)}. + +-spec build_healthcheck(map() | undefined) -> map(). +build_healthcheck(undefined) -> + #{}; +build_healthcheck(Healthcheck) when is_map(Healthcheck) -> + #{ + <<"Test">> => [to_binary(Item) || Item <- field(Healthcheck, <<"test">>, [])], + <<"Interval">> => field(Healthcheck, <<"interval_ns">>, 0), + <<"Timeout">> => field(Healthcheck, <<"timeout_ns">>, 0), + <<"Retries">> => field(Healthcheck, <<"retries">>, 0) + }. + +-spec build_restart(map() | undefined) -> map(). +build_restart(undefined) -> + #{}; +build_restart(RestartPolicy0) when is_map(RestartPolicy0) -> + RestartPolicy = #{ + <<"Name">> => to_binary(field(RestartPolicy0, <<"name">>, <<>>)) + }, + case field(RestartPolicy0, <<"maximum_retry_count">>, 0) of + 0 -> + #{<<"RestartPolicy">> => RestartPolicy}; + RetryCount -> + #{<<"RestartPolicy">> => RestartPolicy#{ + <<"MaximumRetryCount">> => RetryCount + }} + end. + +-spec build_privileged(any()) -> map(). +build_privileged(Privileged) -> + case to_bool(Privileged) of + true -> + #{<<"Privileged">> => true}; + _ -> + #{} + end. + +-spec build_cap_add_drop([binary() | list() | atom()], [binary() | list() | atom()]) -> map(). +build_cap_add_drop(Add, Drop) when is_list(Add), is_list(Drop) -> + case {Add, Drop} of + {[], []} -> + #{}; + _ -> + #{ + <<"CapAdd">> => [to_binary(Item) || Item <- Add], + <<"CapDrop">> => [to_binary(Item) || Item <- Drop] + } + end. + +-spec build_devices([map()]) -> map(). +build_devices(Devices) when is_list(Devices) -> + case Devices of + [] -> + #{}; + _ -> + DevObjs = [#{ + <<"PathOnHost">> => to_binary(HostPath), + <<"PathInContainer">> => to_binary(ContainerPath), + <<"CgroupPermissions">> => device_permissions(Permissions) + } || #{ + <<"path_on_host">> := HostPath, + <<"path_in_container">> := ContainerPath, + <<"cgroup_permissions">> := Permissions + } <- Devices], + #{<<"Devices">> => DevObjs} + end. + +-spec build_resources(integer(), integer(), integer(), integer()) -> map(). +build_resources(MemoryBytes, ReservationBytes, NanoCpus, CpuShares) -> + HostConfig0 = #{}, + HostConfig1 = case MemoryBytes of + 0 -> + HostConfig0; + _ -> + HostConfig0#{<<"Memory">> => MemoryBytes} + end, + HostConfig2 = case ReservationBytes of + 0 -> + HostConfig1; + _ -> + HostConfig1#{<<"MemoryReservation">> => ReservationBytes} + end, + HostConfig3 = case NanoCpus of + 0 -> + HostConfig2; + _ -> + HostConfig2#{<<"NanoCpus">> => NanoCpus} + end, + case CpuShares of + 0 -> + HostConfig3; + _ -> + HostConfig3#{<<"CpuShares">> => CpuShares} + end. + +-spec build_port_bindings([map()]) -> map(). +build_port_bindings(PortBindings) when is_list(PortBindings) -> + case PortBindings of + [] -> + #{}; + _ -> + #{<<"PortBindings">> => fold_port_bindings(PortBindings)} + end. + +-spec fold_port_bindings([map()]) -> map(). +fold_port_bindings(PortBindings) -> + lists:foldl( + fun(PortBinding, Acc) -> + PortKey = port_binding_key(PortBinding), + Binding = port_binding_value(PortBinding), + Existing = maps:get(PortKey, Acc, []), + Acc#{PortKey => Existing ++ [Binding]} + end, + #{}, + PortBindings). + +-spec port_binding_key(map()) -> binary(). +port_binding_key(#{<<"container_port">> := ContainerPort, <<"protocol">> := Protocol}) -> + PortBin = integer_to_binary(ContainerPort), + ProtocolBin = to_binary(Protocol), + case ProtocolBin of + <<>> -> + <>; + <<"tcp">> -> + <>; + _ -> + <> + end. + +-spec port_binding_value(map()) -> map(). +port_binding_value(#{<<"host_port">> := HostPort} = PortBinding) -> + HostIp = to_binary(field(PortBinding, <<"host_ip">>, <<>>)), + #{<<"HostIp">> => HostIp, <<"HostPort">> => integer_to_binary(HostPort)}. + +-spec build_ulimits([map()]) -> map(). +build_ulimits(Ulimits) when is_list(Ulimits) -> + case Ulimits of + [] -> + #{}; + _ -> + #{<<"Ulimits">> => [#{ + <<"Name">> => to_binary(Name), + <<"Soft">> => Soft, + <<"Hard">> => Hard + } || #{<<"name">> := Name, <<"soft">> := Soft, <<"hard">> := Hard} <- Ulimits]} + end. + +-spec build_sysctls(map()) -> map(). +build_sysctls(Sysctls) when is_map(Sysctls) -> + case maps:size(Sysctls) of + 0 -> + #{}; + _ -> + #{<<"Sysctls">> => maps:from_list([{to_binary(Key), to_binary(Value)} || {Key, Value} <- maps:to_list(Sysctls)])} + end. + +-spec build_tmpfs(map()) -> map(). +build_tmpfs(Tmpfs) when is_map(Tmpfs) -> + case maps:size(Tmpfs) of + 0 -> + #{}; + _ -> + #{<<"Tmpfs">> => maps:from_list([{to_binary(Path), to_binary(Options)} || {Path, Options} <- maps:to_list(Tmpfs)])} + end. + +-spec build_extra_hosts([binary() | list() | atom()]) -> map(). +build_extra_hosts(Hosts) when is_list(Hosts) -> + case Hosts of + [] -> + #{}; + _ -> + #{<<"ExtraHosts">> => [to_binary(Host) || Host <- Hosts]} + end. + +-spec normalize_expose_port(map()) -> binary(). +normalize_expose_port(#{<<"container_port">> := Port, <<"protocol">> := Protocol}) -> + PortBin = integer_to_binary(Port), + ProtocolBin = to_binary(Protocol), + case ProtocolBin of + <<>> -> + <>; + <<"tcp">> -> + <>; + _ -> + <> + end. + +-spec device_permissions(binary() | list() | atom()) -> binary(). +device_permissions(<<>>) -> + <<"rwm">>; +device_permissions(Permissions) -> + to_binary(Permissions). + +-spec add_unique_front([binary()], [binary()]) -> [binary()]. +add_unique_front([], List) -> + List; +add_unique_front([Item | Rest], List) -> + NList = case lists:member(Item, List) of + true -> + List; + false -> + [Item | List] + end, + add_unique_front(Rest, NList). + +-spec field(map(), binary(), term()) -> term(). +field(Map, Key, Default) when is_map(Map), is_binary(Key) -> + maps:get(Key, Map, Default). + +-spec to_binary(binary() | list() | atom() | any()) -> binary(). +to_binary(Value) when is_binary(Value) -> + Value; +to_binary(Value) when is_list(Value) -> + list_to_binary(Value); +to_binary(Value) when is_atom(Value) -> + atom_to_binary(Value, utf8); +to_binary(Value) -> + iolist_to_binary(io_lib:format("~p", [Value])). + +-spec to_bool(true | false | 0 | 1 | undefined) -> boolean(). +to_bool(true) -> + true; +to_bool(1) -> + true; +to_bool(_) -> + false. diff --git a/apps/docker/src/docker_deploy_manager.erl b/apps/docker/src/docker_deploy_manager.erl new file mode 100644 index 0000000..9cecaa1 --- /dev/null +++ b/apps/docker/src/docker_deploy_manager.erl @@ -0,0 +1,90 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2026, +%%% @doc +%%% +%%% @end +%%% Created : 20. 4月 2026 +%%%------------------------------------------------------------------- +-module(docker_deploy_manager). +-author("anlicheng"). + +-behaviour(gen_server). + +%% API +-export([start_link/0, deploy/2]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). + +-define(SERVER, ?MODULE). + +-record(state, { + root_dir :: string(), + task_map = #{} +}). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec start_link() -> {ok, pid()} | ignore | {error, term()}. +start_link() -> + gen_server:start_link({local, ?SERVER}, ?MODULE, [], []). + +-spec deploy(integer(), map()) -> ok | {error, binary()}. +deploy(TaskId, Params) when is_integer(TaskId), is_map(Params) -> + gen_server:call(?SERVER, {deploy, TaskId, Params}). + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +-spec init(list()) -> {ok, #state{}}. +init([]) -> + erlang:process_flag(trap_exit, true), + {ok, RootDir} = docker_helper:root_dir(), + {ok, #state{root_dir = RootDir}}. + +-spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, term(), #state{}}. +handle_call({deploy, TaskId, Params}, _From, State = #state{root_dir = RootDir, task_map = TaskMap}) + when is_map(Params) -> + ContainerName = maps:get(<<"container_name">>, Params), + {ok, ContainerDir} = docker_helper:ensure_container_dir(RootDir, ContainerName), + {ok, {TaskPid, _Ref}} = docker_deployer:start_monitor(TaskId, ContainerDir, Params), + logger:debug("[docker_deploy_manager] start deploy task_id: ~p, params: ~p", [TaskId, Params]), + {reply, ok, State#state{task_map = maps:put(TaskPid, TaskId, TaskMap)}}; +handle_call(_Request, _From, State) -> + {reply, ok, State}. + +-spec handle_cast(term(), #state{}) -> {noreply, #state{}}. +handle_cast(_Request, State) -> + {noreply, State}. + +-spec handle_info(term(), #state{}) -> {noreply, #state{}}. +handle_info({'DOWN', _Ref, process, TaskPid, Reason}, State = #state{task_map = TaskMap}) -> + case maps:take(TaskPid, TaskMap) of + error -> + {noreply, State}; + {TaskId, NTaskMap} -> + case Reason of + normal -> + logger:debug("[docker_deploy_manager] task_id: ~p, exit normal", [TaskId]); + Error0 -> + Error = iolist_to_binary(io_lib:format("~p", [Error0])), + docker_task_reporter:stream(TaskId, <<"error">>, <<"任务失败: "/utf8, Error/binary>>), + docker_task_reporter:close(TaskId, <<"task exited">>), + logger:notice("[docker_deploy_manager] task_id: ~p, exit with error: ~p", [TaskId, Error]) + end, + {noreply, State#state{task_map = NTaskMap}} + end; +handle_info(_Info, State) -> + {noreply, State}. + +-spec terminate(term(), #state{}) -> ok. +terminate(_Reason, _State) -> + ok. + +-spec code_change(term(), #state{}, term()) -> {ok, #state{}}. +code_change(_OldVsn, State, _Extra) -> + {ok, State}. diff --git a/apps/docker/src/docker_deployer.erl b/apps/docker/src/docker_deployer.erl new file mode 100644 index 0000000..3ab24b2 --- /dev/null +++ b/apps/docker/src/docker_deployer.erl @@ -0,0 +1,211 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2025, +%%% @doc +%%% +%%% @end +%%% Created : 07. 5月 2025 15:47 +%%%------------------------------------------------------------------- +-module(docker_deployer). +-author("anlicheng"). +-dialyzer([{nowarn_function, normalize_image/1}]). + +%% API +-export([start_monitor/3]). +-export([deploy/3]). + +-define(TASK_SUCCESS, <<"success">>). +-define(TASK_FAIL, <<"fail">>). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec(start_monitor(TaskId :: integer(), ContainerDir :: string(), Params :: map()) -> + {ok, {pid(), reference()}}). +start_monitor(TaskId, ContainerDir, Params) + when is_integer(TaskId), is_list(ContainerDir), is_map(Params) -> + {ok, spawn_monitor(?MODULE, deploy, [TaskId, ContainerDir, Params])}. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== + +%{ +% "image": "nginx:latest", +% "container_name": "my_nginx", +% "ports": ["8080:80", "443:443"], +% "volumes": ["/host/data:/data", "/host/log:/var/log"], +% "envs": ["ENV1=val1", "ENV2=val2"], +% "entrypoint": ["/docker-entrypoint.sh"], +% "command": ["nginx", "-g", "daemon off;"], +% "restart": "always" +%} +-spec deploy(TaskId :: integer(), ContainerDir :: string(), Params :: map()) -> ok. +deploy(TaskId, ContainerDir, Params) when is_integer(TaskId), is_list(ContainerDir), is_map(Params) -> + ContainerName = deploy_container_name(Params), + Image0 = deploy_image(Params), + report_task_event(TaskId, <<"info">>, <<"开始部署容器:"/utf8, ContainerName/binary>>), + try + ok = ensure_container_absent(TaskId, ContainerName), + {ok, Image} = ensure_image_ready(TaskId, Image0), + {ok, ContainerId} = create_container_and_config(TaskId, ContainerDir, Params), + ShortContainerId = short_container_id(ContainerId), + report_task_event(TaskId, <<"info">>, <<"容器创建成功: "/utf8, ShortContainerId/binary>>), + report_task_event(TaskId, <<"info">>, <<"任务完成"/utf8>>), + write_task_summary(TaskId, <<"success">>, ContainerName, Image, ContainerId), + docker_task_reporter:close(TaskId, ?TASK_SUCCESS) + catch + throw:{deploy_error, Reason} -> + report_task_event(TaskId, <<"error">>, Reason), + report_task_event(TaskId, <<"error">>, <<"任务失败"/utf8>>), + write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined), + write_task_failure_reason(TaskId, Reason), + docker_task_reporter:close(TaskId, ?TASK_FAIL); + Class:Reason:Stacktrace -> + Error = iolist_to_binary(io_lib:format("deploy crashed: ~p:~p ~p", [Class, Reason, Stacktrace])), + report_task_event(TaskId, <<"error">>, Error), + report_task_event(TaskId, <<"error">>, <<"任务失败"/utf8>>), + write_task_summary(TaskId, <<"fail">>, ContainerName, Image0, undefined), + write_task_failure_reason(TaskId, Error), + docker_task_reporter:close(TaskId, ?TASK_FAIL) + end. + +-spec normalize_image(binary()) -> binary(). +normalize_image(Image) when is_binary(Image) -> + Parts = binary:split(Image, <<"/">>, [global]), + {PrefixParts, [Last]} = lists:split(length(Parts) - 1, Parts), + NormalizedLast = case binary:split(Last, <<":">>, [global]) of + [_Name] -> <>; + [_Name, _Tag] -> Last + end, + iolist_to_binary(lists:join(<<"/">>, PrefixParts ++ [NormalizedLast])). + +-spec report_task_event(TaskId :: integer(), Level :: binary(), Msg :: binary()) -> ok. +report_task_event(TaskId, Level, Msg) when is_integer(TaskId), is_binary(Level), is_binary(Msg) -> + docker_task_reporter:stream(TaskId, Level, Msg). + +-spec write_task_summary(integer(), binary(), binary(), binary(), undefined | binary()) -> ok. +write_task_summary(TaskId, Status, ContainerName, Image, ContainerId) + when is_integer(TaskId), is_binary(Status), is_binary(ContainerName), is_binary(Image) -> + Fields0 = [ + <<"type=deploy_summary">>, + <<"task_id=">>, integer_to_binary(TaskId), + <<" status=">>, Status, + <<" container_name=">>, ContainerName, + <<" image=">>, Image + ], + Fields = case ContainerId of + undefined -> + Fields0; + ContainerId0 when is_binary(ContainerId0) -> + Fields0 ++ [<<" container_id=">>, short_container_id(ContainerId0)] + end, + efka_logger:write(iolist_to_binary(Fields)). + +-spec write_task_failure_reason(integer(), binary()) -> ok. +write_task_failure_reason(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) -> + Info = iolist_to_binary([ + <<"type=deploy_failure_reason task_id=">>, + integer_to_binary(TaskId), + <<" reason=">>, + Reason + ]), + efka_logger:write(Info). + +-spec ensure_container_absent(TaskId :: integer(), ContainerName :: binary()) -> ok. +ensure_container_absent(TaskId, ContainerName) when is_integer(TaskId), is_binary(ContainerName) -> + report_task_event(TaskId, <<"info">>, <<"开始创建容器: "/utf8, ContainerName/binary>>), + ok. + +-spec ensure_image_ready(TaskId :: integer(), Image0 :: binary()) -> {ok, binary()}. +ensure_image_ready(TaskId, Image0) when is_integer(TaskId), is_binary(Image0) -> + Image = normalize_image(Image0), + report_task_event(TaskId, <<"info">>, <<"使用镜像:"/utf8, Image/binary>>), + report_task_event(TaskId, <<"info">>, <<"开始拉取镜像:"/utf8, Image/binary>>), + case docker_commands:pull_image(Image) of + {ok, Ref, Pid, MRef} -> + await_pull_image(TaskId, Ref, Pid, MRef), + {ok, Image} + end. + +-spec await_pull_image(TaskId :: integer(), Ref :: reference(), Pid :: pid(), MRef :: reference()) -> ok. +await_pull_image(TaskId, Ref, Pid, MRef) -> + receive + {docker_client, Ref, {response, _Status, _Headers}} -> + await_pull_image(TaskId, Ref, Pid, MRef); + {docker_client, Ref, {data, Data}} -> + report_task_event(TaskId, <<"info">>, Data), + await_pull_image(TaskId, Ref, Pid, MRef); + {docker_client, Ref, done} -> + erlang:demonitor(MRef, [flush]), + ok; + {docker_client, Ref, {error, Reason}} -> + erlang:demonitor(MRef, [flush]), + report_task_event(TaskId, <<"error">>, Reason), + throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>}); + {'DOWN', MRef, process, Pid, Reason0} -> + Reason = iolist_to_binary(io_lib:format("~p", [Reason0])), + throw({deploy_error, <<"镜像拉取失败: "/utf8, Reason/binary>>}) + end. + +-spec create_container_and_config(TaskId :: integer(), ContainerDir :: string(), Params :: map()) -> + {ok, binary()}. +create_container_and_config(TaskId, ContainerDir, Params) + when is_integer(TaskId), is_list(ContainerDir), is_map(Params) -> + case docker_commands:create_container(ContainerDir, Params) of + {ok, ContainerId} -> + ok = create_config_file(TaskId, ContainerDir), + {ok, ContainerId}; + {error, Reason} when is_binary(Reason) -> + throw({deploy_error, format_create_container_error(Reason)}); + {error, Reason} -> + Error = iolist_to_binary(io_lib:format("容器创建失败: ~p", [Reason])), + throw({deploy_error, Error}) + end. + +-spec create_config_file(TaskId :: integer(), ContainerDir :: string()) -> ok. +create_config_file(TaskId, ContainerDir) when is_integer(TaskId), is_list(ContainerDir) -> + ConfigFile = docker_helper:get_config_file(ContainerDir), + case file:open(ConfigFile, [write, exclusive]) of + {ok, FD} -> + ok = file:write(FD, <<>>), + file:close(FD), + ok; + {error, Reason} -> + ReasonBin = list_to_binary(io_lib:format("~p", [Reason])), + report_task_event(TaskId, <<"notice">>, <<"创建配置文件失败: "/utf8, ReasonBin/binary>>), + ok + end. + +-spec format_create_container_error(binary()) -> binary(). +format_create_container_error(Reason) when is_binary(Reason) -> + case is_container_already_exists_error(Reason) of + true -> + <<"本地容器已经存在"/utf8>>; + false -> + <<"容器创建失败: "/utf8, Reason/binary>> + end. + +-spec is_container_already_exists_error(binary()) -> boolean(). +is_container_already_exists_error(Reason) when is_binary(Reason) -> + binary:match(Reason, <<"is already in use by container">>) =/= nomatch orelse + binary:match(Reason, <<"Conflict. The container name ">>) =/= nomatch. + +-spec short_container_id(binary()) -> binary(). +short_container_id(ContainerId) when is_binary(ContainerId), byte_size(ContainerId) >= 12 -> + binary:part(ContainerId, 0, 12); +short_container_id(ContainerId) when is_binary(ContainerId) -> + ContainerId. + +-spec deploy_container_name(map()) -> binary(). +deploy_container_name(#{<<"container_name">> := ContainerName}) when is_binary(ContainerName) -> + ContainerName; +deploy_container_name(_) -> + throw({deploy_error, <<"invalid deploy params: container_name missing">>}). + +-spec deploy_image(map()) -> binary(). +deploy_image(#{<<"create">> := #{<<"config">> := #{<<"image">> := Image}}}) when is_binary(Image) -> + Image; +deploy_image(_) -> + throw({deploy_error, <<"invalid deploy params: image missing">>}). diff --git a/apps/efka/src/models/cache_model.erl b/apps/docker/src/docker_events.erl similarity index 55% rename from apps/efka/src/models/cache_model.erl rename to apps/docker/src/docker_events.erl index 4a519c6..62c1768 100644 --- a/apps/efka/src/models/cache_model.erl +++ b/apps/docker/src/docker_events.erl @@ -4,46 +4,35 @@ %%% @doc %%% %%% @end -%%% Created : 13. 8月 2025 16:05 +%%% Created : 16. 9月 2025 16:48 %%%------------------------------------------------------------------- --module(cache_model). +-module(docker_events). -author("anlicheng"). --include("efka_tables.hrl"). -behaviour(gen_server). %% API -export([start_link/0]). --export([insert/2, fetch_next/0, delete/1, get_all_cache/0]). +-export([monitor_container/2]). %% gen_server callbacks -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). -define(SERVER, ?MODULE). --define(TAB, cache). -record(state, { - + port, + %% 观察者 + monitors = #{} }). %%%=================================================================== %%% API %%%=================================================================== --spec insert(Method :: integer(), Data :: binary()) -> ok | {error, Reason :: any()}. -insert(Method, Data) when is_integer(Method), is_binary(Data) -> - Cache = #cache{id = generate_id(), method = Method, data = Data}, - gen_server:call(?SERVER, {insert, Cache}). - -fetch_next() -> - gen_server:call(?SERVER, fetch_next). - -delete(Id) when is_integer(Id) -> - gen_server:call(?SERVER, {delete, Id}). - --spec get_all_cache() -> [#cache{}]. -get_all_cache() -> - gen_server:call(?SERVER, get_all_cache). +-spec monitor_container(ReceiverPid :: pid(), ContainerId :: binary()) -> ok. +monitor_container(ReceiverPid, ContainerId) when is_pid(ReceiverPid), is_binary(ContainerId) -> + gen_server:cast(?SERVER, {monitor_container, ReceiverPid, ContainerId}). %% @doc Spawns the server and registers the local name (unique) -spec(start_link() -> @@ -61,9 +50,8 @@ start_link() -> {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | {stop, Reason :: term()} | ignore). init([]) -> - {ok, DetsDir} = application:get_env(efka, dets_dir), - File = DetsDir ++ "cache.dets", - {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, bag}, {keypos, 2}]), + process_flag(trap_exit, true), + try_attach_events(0), {ok, #state{}}. %% @private @@ -76,25 +64,6 @@ init([]) -> {noreply, NewState :: #state{}, timeout() | hibernate} | {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | {stop, Reason :: term(), NewState :: #state{}}). -handle_call({insert, Cache}, _From, State = #state{}) -> - ok = dets:insert(?TAB, Cache), - {reply, ok, State}; -handle_call(fetch_next, _From, State = #state{}) -> - case dets:first(?TAB) of - '$end_of_table' -> - {reply, error, State}; - Key -> - [Entry] = dets:lookup(?TAB, Key), - {reply, {ok, Entry}, State} - end; -handle_call({delete, Id}, _From, State = #state{}) -> - ok = dets:delete(?TAB, Id), - {reply, ok, State}; - -handle_call(get_all_cache, _From, State = #state{}) -> - Items = dets:foldl(fun(Record, Acc) -> [Record|Acc] end, [], ?TAB), - {reply, {ok, lists:reverse(Items)}, State}; - handle_call(_Request, _From, State = #state{}) -> {reply, ok, State}. @@ -104,8 +73,9 @@ handle_call(_Request, _From, State = #state{}) -> {noreply, NewState :: #state{}} | {noreply, NewState :: #state{}, timeout() | hibernate} | {stop, Reason :: term(), NewState :: #state{}}). -handle_cast(_Request, State = #state{}) -> - {noreply, State}. +handle_cast({monitor_container, ReceiverPid, ContainerId}, State = #state{monitors = Monitors}) -> + MRef = erlang:monitor(process, ReceiverPid), + {noreply, State#state{monitors = maps:put(ContainerId, {ReceiverPid, MRef}, Monitors)}}. %% @private %% @doc Handling all non call/cast messages @@ -113,8 +83,31 @@ handle_cast(_Request, State = #state{}) -> {noreply, NewState :: #state{}} | {noreply, NewState :: #state{}, timeout() | hibernate} | {stop, Reason :: term(), NewState :: #state{}}). -handle_info(_Info, State = #state{}) -> - {noreply, State}. +handle_info({timeout, _, attach_docker_events}, State = #state{port = undefined}) -> + ExecCmd = "docker events --format \"{{json .}}\"", + case catch erlang:open_port({spawn, ExecCmd}, [exit_status, {line, 10239}, use_stdio, stderr_to_stdout, binary]) of + Port when is_port(Port) -> + {noreply, State#state{port = Port}}; + _Error -> + try_attach_events(5000), + {noreply, State} + end; +handle_info({Port, {data, {eol, BinLine}}}, State = #state{port = Port}) -> + Event = catch json:decode(BinLine), + logger:debug("event: ~p", [Event]), + handle_event(Event, State), + {noreply, State}; + +%% 进程退出的时候删除掉管理的Pid +handle_info({'DOWN', MRef, process, _Pid, _Reason}, State = #state{monitors = Monitors}) -> + NMonitors = maps:filter(fun(_Key, {_, Ref}) -> MRef =/= Ref end, Monitors), + {noreply, State#state{monitors = NMonitors}}; + +%% Port退出的时候,尝试重启 +handle_info({'EXIT', Port, Reason}, State = #state{port = Port}) -> + logger:warning("[efka_docker_events] exit with reason: ~p", [Reason]), + try_attach_events(5000), + {noreply, State#state{port = undefined}}. %% @private %% @doc This function is called by a gen_server when it is about to @@ -124,7 +117,6 @@ handle_info(_Info, State = #state{}) -> -spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), State :: #state{}) -> term()). terminate(_Reason, _State = #state{}) -> - dets:close(?TAB), ok. %% @private @@ -138,7 +130,24 @@ code_change(_OldVsn, State = #state{}, _Extra) -> %%%=================================================================== %%% Internal functions %%%=================================================================== +-spec handle_event(term(), #state{}) -> ok. +handle_event(#{<<"Type">> := <<"container">>, <<"status">> := Status, <<"id">> := Id}, #state{monitors = Monitors}) -> + case maps:find(Id, Monitors) of + error -> + ok; + {ok, {ReceiverPid, _}} -> + case Status of + <<"start">> -> + ReceiverPid ! {docker_events, start}; + <<"stop">> -> + ReceiverPid ! {docker_events, stop}; + _ -> + ok + end + end; +handle_event(_, _) -> + ok. --spec generate_id() -> integer(). -generate_id() -> - os:system_time(microsecond). \ No newline at end of file +-spec try_attach_events(non_neg_integer()) -> reference(). +try_attach_events(Timeout) -> + erlang:start_timer(Timeout, self(), attach_docker_events). diff --git a/apps/docker/src/docker_helper.erl b/apps/docker/src/docker_helper.erl new file mode 100644 index 0000000..48c7aa9 --- /dev/null +++ b/apps/docker/src/docker_helper.erl @@ -0,0 +1,102 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2025, +%%% @doc +%%% +%%% @end +%%% Created : 17. 9月 2025 14:50 +%%%------------------------------------------------------------------- +-module(docker_helper). +-author("anlicheng"). + +%% API +-export([root_dir/0]). +-export([ensure_container_dir/2, ensure_container_dir/3, get_container_dir/2, get_config_file/1]). +-export([update_container_config/2]). + +-spec root_dir() -> {ok, string()} | undefined. +root_dir() -> + case application:get_env(docker, root_dir) of + {ok, RootDir} -> + {ok, RootDir}; + undefined -> + application:get_env(efka, root_dir) + end. + +-spec ensure_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()}. +ensure_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) -> + ensure_container_dir(RootDir, ContainerName, <<>>). + +-spec ensure_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) -> + {ok, ServerRootDir :: string()}. +ensure_container_dir(RootDir, ContainerName, ContainerDir) when is_list(RootDir), is_binary(ContainerName), is_binary(ContainerDir) -> + PointerDir = default_container_dir(RootDir, ContainerName), + PointerFile = container_dir_pointer_file(PointerDir), + ActualContainerDir = resolve_container_dir(RootDir, ContainerName, ContainerDir), + ok = filelib:ensure_dir(PointerDir), + ok = filelib:ensure_dir(ActualContainerDir), + ok = file:write_file(PointerFile, unicode:characters_to_binary(ActualContainerDir), [write]), + {ok, ActualContainerDir}. + +-spec get_config_file(ContainerDir :: string()) -> ConfigFile :: string(). +get_config_file(ContainerDir) when is_list(ContainerDir) -> + %% 根目录 + ContainerDir ++ "service.conf". + +-spec get_container_dir(RootDir :: string(), ContainerName :: binary()) -> {ok, ServerRootDir :: string()} | error. +get_container_dir(RootDir, ContainerName) when is_list(RootDir), is_binary(ContainerName) -> + ContainerRootDir = default_container_dir(RootDir, ContainerName), + PointerFile = container_dir_pointer_file(ContainerRootDir), + ResolvedContainerDir = case file:read_file(PointerFile) of + {ok, ActualContainerDirBin} -> + normalize_container_dir(binary_to_list(ActualContainerDirBin)); + {error, _} -> + ContainerRootDir + end, + case filelib:is_dir(ResolvedContainerDir) of + true -> + {ok, ResolvedContainerDir}; + false -> + error + end. + +-spec update_container_config(binary(), binary()) -> ok | {error, binary()}. +update_container_config(ContainerName, Config) when is_binary(ContainerName), is_binary(Config) -> + {ok, RootDir} = root_dir(), + case get_container_dir(RootDir, ContainerName) of + {ok, ContainerDir} -> + ConfigFile = get_config_file(ContainerDir), + case file:write_file(ConfigFile, Config, [write, binary]) of + ok -> + logger:warning("[efka_iot_client] write config file: ~p success", [ConfigFile]), + ok; + {error, Reason} -> + logger:warning("[efka_iot_client] write config file: ~p, get error: ~p", [ConfigFile, Reason]), + {error, <<"write config failed">>} + end; + error -> + {error, <<"error">>} + end. + +-spec default_container_dir(RootDir :: string(), ContainerName :: binary()) -> string(). +default_container_dir(RootDir, ContainerName) -> + normalize_container_dir(RootDir ++ "/" ++ binary_to_list(ContainerName)). + +-spec container_dir_pointer_file(ContainerDir :: string()) -> string(). +container_dir_pointer_file(ContainerDir) -> + ContainerDir ++ ".container_dir". + +-spec resolve_container_dir(RootDir :: string(), ContainerName :: binary(), ContainerDir :: binary()) -> string(). +resolve_container_dir(RootDir, ContainerName, <<>>) -> + default_container_dir(RootDir, ContainerName); +resolve_container_dir(_RootDir, _ContainerName, ContainerDir) -> + normalize_container_dir(binary_to_list(ContainerDir)). + +-spec normalize_container_dir(ContainerDir :: string()) -> string(). +normalize_container_dir(ContainerDir) -> + case lists:last(ContainerDir) of + $/ -> + ContainerDir; + _ -> + ContainerDir ++ "/" + end. diff --git a/apps/docker/src/docker_sup.erl b/apps/docker/src/docker_sup.erl new file mode 100644 index 0000000..2fd5ee3 --- /dev/null +++ b/apps/docker/src/docker_sup.erl @@ -0,0 +1,42 @@ +%%%------------------------------------------------------------------- +%% @doc Docker top level supervisor. +%% @end +%%%------------------------------------------------------------------- + +-module(docker_sup). + +-behaviour(supervisor). + +-export([start_link/0]). +-export([init/1]). + +-define(SERVER, ?MODULE). + +-spec start_link() -> {ok, pid()} | ignore | {error, term()}. +start_link() -> + supervisor:start_link({local, ?SERVER}, ?MODULE, []). + +-spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}. +init([]) -> + SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600}, + ChildSpecs = [ + #{ + id => docker_task_reporter, + start => {docker_task_reporter, start_link, []}, + restart => permanent, + shutdown => 2000, + type => worker, + modules => [docker_task_reporter] + }, + + #{ + id => docker_deploy_manager, + start => {docker_deploy_manager, start_link, []}, + restart => permanent, + shutdown => 2000, + type => worker, + modules => [docker_deploy_manager] + } + ], + + {ok, {SupFlags, ChildSpecs}}. diff --git a/apps/docker/src/docker_task_reporter.erl b/apps/docker/src/docker_task_reporter.erl new file mode 100644 index 0000000..897f1dd --- /dev/null +++ b/apps/docker/src/docker_task_reporter.erl @@ -0,0 +1,121 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2026, +%%% @doc +%%% +%%% @end +%%% Created : 20. 4月 2026 +%%%------------------------------------------------------------------- +-module(docker_task_reporter). +-author("anlicheng"). + +-behaviour(gen_server). + +%% API +-export([start_link/0]). +-export([stream/3, close/2]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). + +-define(SERVER, ?MODULE). +-define(FLUSH_INTERVAL, 1000). + +-record(state, { + pending = queue:new(), + flush_ref = undefined +}). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec stream(TaskId :: integer(), Type :: binary(), Stream :: binary()) -> ok. +stream(TaskId, Type, Stream) when is_integer(TaskId), is_binary(Type), is_binary(Stream) -> + gen_server:cast(?SERVER, {stream, TaskId, Type, Stream}). + +-spec close(TaskId :: integer(), Reason :: binary()) -> ok. +close(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) -> + gen_server:cast(?SERVER, {close, TaskId, Reason}). + +-spec start_link() -> {ok, pid()} | ignore | {error, term()}. +start_link() -> + gen_server:start_link({local, ?SERVER}, ?MODULE, [], []). + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +-spec init(list()) -> {ok, #state{}}. +init([]) -> + {ok, #state{}}. + +-spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, ok, #state{}}. +handle_call(_Request, _From, State) -> + {reply, ok, State}. + +-spec handle_cast(term(), #state{}) -> {noreply, #state{}}. +handle_cast({stream, TaskId, Type, Stream}, State0 = #state{pending = Pending0}) -> + Pending = queue:in({stream, TaskId, Type, Stream}, Pending0), + {noreply, ensure_flush_timer(State0#state{pending = Pending}, 0)}; +handle_cast({close, TaskId, Reason}, State0 = #state{pending = Pending0}) -> + Pending = queue:in({close, TaskId, Reason}, Pending0), + {noreply, ensure_flush_timer(State0#state{pending = Pending}, 0)}; +handle_cast(_Request, State) -> + {noreply, State}. + +-spec handle_info(term(), #state{}) -> {noreply, #state{}}. +handle_info(flush, State0 = #state{}) -> + State1 = State0#state{flush_ref = undefined}, + {noreply, flush_pending(State1)}; +handle_info(_Info, State) -> + {noreply, State}. + +-spec terminate(term(), #state{}) -> ok. +terminate(_Reason, _State) -> + ok. + +-spec code_change(term(), #state{}, term()) -> {ok, #state{}}. +code_change(_OldVsn, State, _Extra) -> + {ok, State}. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== + +-spec ensure_flush_timer(#state{}, non_neg_integer()) -> #state{}. +ensure_flush_timer(State = #state{flush_ref = undefined}, Delay) when is_integer(Delay), Delay >= 0 -> + Ref = erlang:send_after(Delay, self(), flush), + State#state{flush_ref = Ref}; +ensure_flush_timer(State = #state{}, _Delay) -> + State. + +-spec flush_pending(#state{}) -> #state{}. +flush_pending(State = #state{pending = Pending0}) -> + case queue:out(Pending0) of + {empty, _} -> + State; + {{value, Event}, Pending1} -> + case send_event(Event) of + ok -> + flush_pending(State#state{pending = Pending1}); + not_ready -> + ensure_flush_timer(State#state{pending = Pending0}, ?FLUSH_INTERVAL) + end + end. + +-spec send_event({stream, integer(), binary(), binary()} | {close, integer(), binary()}) -> ok | not_ready. +send_event({stream, TaskId, Type, Stream}) -> + maybe_send(fun() -> efka_iot_client:task_event_stream(TaskId, Type, Stream) end); +send_event({close, TaskId, Reason}) -> + maybe_send(fun() -> efka_iot_client:close_task_event_stream(TaskId, Reason) end). + +-spec maybe_send(fun(() -> any())) -> ok | not_ready. +maybe_send(SendFun) -> + case catch efka_iot_client:is_activated() of + true -> + _ = catch SendFun(), + ok; + _ -> + not_ready + end. diff --git a/apps/docker/test/efka_docker_container_builder_tests.erl b/apps/docker/test/efka_docker_container_builder_tests.erl new file mode 100644 index 0000000..dacc594 --- /dev/null +++ b/apps/docker/test/efka_docker_container_builder_tests.erl @@ -0,0 +1,35 @@ +-module(efka_docker_container_builder_tests). + +-include_lib("eunit/include/eunit.hrl"). + +port_bindings_are_rendered_for_docker_create_test() -> + Name = <<"nginx">>, + ContainerDir = "/tmp/efka-nginx/", + Create = #{ + <<"config">> => #{ + <<"image">> => <<"nginx:latest">>, + <<"exposed_ports">> => [ + #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>}, + #{<<"container_port">> => 443, <<"protocol">> => <<"tcp">>} + ] + }, + <<"host_config">> => #{ + <<"port_bindings">> => [ + #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>}, + #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>} + ] + } + }, + Options = docker_container_builder:build_options(Name, ContainerDir, Create), + ?assertMatch(#{ + <<"ExposedPorts">> := #{ + <<"80/tcp">> := #{}, + <<"443/tcp">> := #{} + }, + <<"HostConfig">> := #{ + <<"PortBindings">> := #{ + <<"80/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"8080">>}], + <<"443/tcp">> := [#{<<"HostIp">> := <<>>, <<"HostPort">> := <<"443">>}] + } + } + }, Options). diff --git a/apps/efka/include/efka.hrl b/apps/efka/include/efka.hrl deleted file mode 100644 index fcd8c1f..0000000 --- a/apps/efka/include/efka.hrl +++ /dev/null @@ -1,54 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% 扩展部分, 1: 支持基于topic的pub/sub机制; 2. 基于target的单点通讯和广播 -%%% @end -%%% Created : 21. 4月 2025 17:28 -%%%------------------------------------------------------------------- --author("anlicheng"). - -%% efka主动发起的消息体类型, 消息大类 --define(PACKET_REQUEST, 16#01). --define(PACKET_RESPONSE, 16#02). - -%% 服务器基于pub/sub的消息, 消息大类 --define(PACKET_PUB, 16#03). - -%% push调用不需要返回, 消息大类 --define(PACKET_COMMAND, 16#04). - -%% 服务器端推送消息 --define(PACKET_ASYNC_CALL, 16#05). --define(PACKET_ASYNC_CALL_REPLY, 16#06). - -%% ping包,客户端主动发起 --define(PACKET_PING, 16#FF). - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%%%% 二级分类定义 -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -%% 主机端上报数据类型标识 --define(METHOD_AUTH, 16#01). --define(METHOD_DATA, 16#02). --define(METHOD_PING, 16#03). --define(METHOD_INFORM, 16#04). --define(METHOD_EVENT, 16#05). --define(METHOD_PHASE, 16#06). --define(METHOD_REQUEST_SERVICE_CONFIG, 16#07). - -%%%% 命令类型子分类, 不需要返回值 - -%% 授权 --define(COMMAND_AUTH, 16#08). - -%%%% 主动推送的消息类型子分类, 需要返回值 - --define(PUSH_DEPLOY, 16#01). --define(PUSH_START_SERVICE, 16#02). --define(PUSH_STOP_SERVICE, 16#03). - --define(PUSH_SERVICE_CONFIG, 16#04). --define(PUSH_INVOKE, 16#05). --define(PUSH_TASK_LOG, 16#06). diff --git a/apps/efka/include/efka_service_pb.hrl b/apps/efka/include/efka_service_pb.hrl new file mode 100644 index 0000000..af290d0 --- /dev/null +++ b/apps/efka/include/efka_service_pb.hrl @@ -0,0 +1,72 @@ +%% -*- coding: utf-8 -*- +%% Automatically generated, do not edit +%% Generated by gpb_compile version 4.21.7 + +-ifndef(efka_service_pb). +-define(efka_service_pb, true). + +-define(efka_service_pb_gpb_version, "4.21.7"). + + +-ifndef('SERVICEREQUEST.REGISTER_PB_H'). +-define('SERVICEREQUEST.REGISTER_PB_H', true). +-record('ServiceRequest.Register', + {service_id = <<>> :: unicode:chardata() | undefined % = 1, optional + }). +-endif. + +-ifndef('SERVICEREQUEST.SUBSCRIBE_PB_H'). +-define('SERVICEREQUEST.SUBSCRIBE_PB_H', true). +-record('ServiceRequest.Subscribe', + {topic = <<>> :: unicode:chardata() | undefined % = 1, optional + }). +-endif. + +-ifndef('SERVICEREQUEST_PB_H'). +-define('SERVICEREQUEST_PB_H', true). +-record('ServiceRequest', + {packet_id = 0 :: non_neg_integer() | undefined, % = 1, optional, 32 bits + request :: {register, efka_service_pb:'ServiceRequest.Register'()} | {subscribe, efka_service_pb:'ServiceRequest.Subscribe'()} | undefined % oneof + }). +-endif. + +-ifndef('SERVICEREPLY.ERROR_PB_H'). +-define('SERVICEREPLY.ERROR_PB_H', true). +-record('ServiceReply.Error', + {code = 0 :: integer() | undefined, % = 1, optional, 32 bits + message = <<>> :: unicode:chardata() | undefined % = 2, optional + }). +-endif. + +-ifndef('SERVICEREPLY_PB_H'). +-define('SERVICEREPLY_PB_H', true). +-record('ServiceReply', + {packet_id = 0 :: non_neg_integer() | undefined, % = 1, optional, 32 bits + reply :: {result, iodata()} | {error, efka_service_pb:'ServiceReply.Error'()} | undefined % oneof + }). +-endif. + +-ifndef('SERVICECAST.METRICDATA_PB_H'). +-define('SERVICECAST.METRICDATA_PB_H', true). +-record('ServiceCast.MetricData', + {route_key = <<>> :: iodata() | undefined, % = 1, optional + metric = <<>> :: iodata() | undefined % = 2, optional + }). +-endif. + +-ifndef('SERVICECAST.TOPICEVENT_PB_H'). +-define('SERVICECAST.TOPICEVENT_PB_H', true). +-record('ServiceCast.TopicEvent', + {topic = <<>> :: unicode:chardata() | undefined, % = 1, optional + content = <<>> :: iodata() | undefined % = 2, optional + }). +-endif. + +-ifndef('SERVICECAST_PB_H'). +-define('SERVICECAST_PB_H', true). +-record('ServiceCast', + {body :: {topic_event, efka_service_pb:'ServiceCast.TopicEvent'()} | {metric_data, efka_service_pb:'ServiceCast.MetricData'()} | undefined % oneof + }). +-endif. + +-endif. diff --git a/apps/efka/include/efka_tables.hrl b/apps/efka/include/efka_tables.hrl index 895d1b6..2892dba 100644 --- a/apps/efka/include/efka_tables.hrl +++ b/apps/efka/include/efka_tables.hrl @@ -4,37 +4,21 @@ %%% @doc %%% %%% @end -%%% Created : 30. 4月 2025 11:16 +%%% Created : 29. 9月 2025 15:27 %%%------------------------------------------------------------------- -author("anlicheng"). +-define(SERVICE_STOPPED, 0). +-define(SERVICE_RUNNING, 1). + %% 用来保存微服务 -record(service, { service_id :: binary(), - tar_url :: binary(), - %% 工作目录 - root_dir :: string(), - %% 配置信息 - config_json :: binary(), + container_name :: binary(), + %% 配置信息, 微服务启动的时候自己注册的信息 + meta_data = #{} :: map(), %% 状态: 0: 停止, 1: 运行中 - status = 0 -}). - -%% 数据缓存 --record(cache, { - id = 0 :: integer(), - method :: integer(), - data :: binary() -}). - -%% 数据缓存 --record(task_log, { - task_id = 0 :: integer(), - logs = []:: list() -}). - -%% id生成器 --record(id_generator, { - id, - value = 1 + status = 0, + create_ts = 0 :: integer(), + update_ts = 0 :: integer() }). \ No newline at end of file diff --git a/apps/efka/include/message_pb.hrl b/apps/efka/include/message_pb.hrl deleted file mode 100644 index a632950..0000000 --- a/apps/efka/include/message_pb.hrl +++ /dev/null @@ -1,128 +0,0 @@ -%% -*- coding: utf-8 -*- -%% Automatically generated, do not edit -%% Generated by gpb_compile version 4.21.1 - --ifndef(message_pb). --define(message_pb, true). - --define(message_pb_gpb_version, "4.21.1"). - - --ifndef('AUTH_REQUEST_PB_H'). --define('AUTH_REQUEST_PB_H', true). --record(auth_request, - {uuid = <<>> :: unicode:chardata() | undefined, % = 1, optional - username = <<>> :: unicode:chardata() | undefined, % = 2, optional - salt = <<>> :: unicode:chardata() | undefined, % = 4, optional - token = <<>> :: unicode:chardata() | undefined, % = 5, optional - timestamp = 0 :: non_neg_integer() | undefined % = 6, optional, 32 bits - }). --endif. - --ifndef('AUTH_REPLY_PB_H'). --define('AUTH_REPLY_PB_H', true). --record(auth_reply, - {code = 0 :: non_neg_integer() | undefined, % = 1, optional, 32 bits - message = <<>> :: unicode:chardata() | undefined % = 2, optional - }). --endif. - --ifndef('PUB_PB_H'). --define('PUB_PB_H', true). --record(pub, - {topic = <<>> :: unicode:chardata() | undefined, % = 1, optional - content = <<>> :: unicode:chardata() | undefined % = 2, optional - }). --endif. - --ifndef('ASYNC_CALL_REPLY_PB_H'). --define('ASYNC_CALL_REPLY_PB_H', true). --record(async_call_reply, - {code = 0 :: non_neg_integer() | undefined, % = 1, optional, 32 bits - result = <<>> :: unicode:chardata() | undefined, % = 2, optional - message = <<>> :: unicode:chardata() | undefined % = 3, optional - }). --endif. - --ifndef('DEPLOY_PB_H'). --define('DEPLOY_PB_H', true). --record(deploy, - {task_id = 0 :: non_neg_integer() | undefined, % = 1, optional, 32 bits - service_id = <<>> :: unicode:chardata() | undefined, % = 2, optional - tar_url = <<>> :: unicode:chardata() | undefined % = 3, optional - }). --endif. - --ifndef('FETCH_TASK_LOG_PB_H'). --define('FETCH_TASK_LOG_PB_H', true). --record(fetch_task_log, - {task_id = 0 :: non_neg_integer() | undefined % = 1, optional, 32 bits - }). --endif. - --ifndef('INVOKE_PB_H'). --define('INVOKE_PB_H', true). --record(invoke, - {service_id = <<>> :: unicode:chardata() | undefined, % = 1, optional - payload = <<>> :: unicode:chardata() | undefined, % = 2, optional - timeout = 0 :: non_neg_integer() | undefined % = 3, optional, 32 bits - }). --endif. - --ifndef('PUSH_SERVICE_CONFIG_PB_H'). --define('PUSH_SERVICE_CONFIG_PB_H', true). --record(push_service_config, - {service_id = <<>> :: unicode:chardata() | undefined, % = 1, optional - config_json = <<>> :: unicode:chardata() | undefined, % = 2, optional - timeout = 0 :: non_neg_integer() | undefined % = 3, optional, 32 bits - }). --endif. - --ifndef('DATA_PB_H'). --define('DATA_PB_H', true). --record(data, - {service_id = <<>> :: unicode:chardata() | undefined, % = 1, optional - device_uuid = <<>> :: unicode:chardata() | undefined, % = 2, optional - metric = <<>> :: unicode:chardata() | undefined % = 3, optional - }). --endif. - --ifndef('PING_PB_H'). --define('PING_PB_H', true). --record(ping, - {adcode = <<>> :: unicode:chardata() | undefined, % = 1, optional - boot_time = 0 :: non_neg_integer() | undefined, % = 2, optional, 32 bits - province = <<>> :: unicode:chardata() | undefined, % = 3, optional - city = <<>> :: unicode:chardata() | undefined, % = 4, optional - efka_version = <<>> :: unicode:chardata() | undefined, % = 5, optional - kernel_arch = <<>> :: unicode:chardata() | undefined, % = 6, optional - ips = [] :: [unicode:chardata()] | undefined, % = 7, repeated - cpu_core = 0 :: non_neg_integer() | undefined, % = 8, optional, 32 bits - cpu_load = 0 :: non_neg_integer() | undefined, % = 9, optional, 32 bits - cpu_temperature = 0.0 :: float() | integer() | infinity | '-infinity' | nan | undefined, % = 10, optional - disk = [] :: [integer()] | undefined, % = 11, repeated, 32 bits - memory = [] :: [integer()] | undefined, % = 12, repeated, 32 bits - interfaces = <<>> :: unicode:chardata() | undefined % = 13, optional - }). --endif. - --ifndef('SERVICE_INFORM_PB_H'). --define('SERVICE_INFORM_PB_H', true). --record(service_inform, - {service_id = <<>> :: unicode:chardata() | undefined, % = 1, optional - props = <<>> :: unicode:chardata() | undefined, % = 2, optional - status = 0 :: non_neg_integer() | undefined, % = 3, optional, 32 bits - timestamp = 0 :: non_neg_integer() | undefined % = 4, optional, 32 bits - }). --endif. - --ifndef('EVENT_PB_H'). --define('EVENT_PB_H', true). --record(event, - {service_id = <<>> :: unicode:chardata() | undefined, % = 1, optional - event_type = 0 :: non_neg_integer() | undefined, % = 2, optional, 32 bits - params = <<>> :: unicode:chardata() | undefined % = 3, optional - }). --endif. - --endif. diff --git a/apps/efka/proto/service.proto b/apps/efka/proto/service.proto new file mode 100644 index 0000000..96d7a61 --- /dev/null +++ b/apps/efka/proto/service.proto @@ -0,0 +1,54 @@ +syntax = "proto3"; + +// 微服务通过 ws_channel 与 efka 之间的通道消息定义。 +// 该协议用于替代当前基于 JSON 的请求/响应/推送格式。 + +message ServiceRequest { + uint32 packet_id = 1; + + message Register { + string service_id = 1; + } + + message Subscribe { + string topic = 1; + } + + oneof request { + Register register = 10; + Subscribe subscribe = 11; + } +} + +message ServiceReply { + uint32 packet_id = 1; + + message Error { + int32 code = 1; + string message = 2; + } + + oneof reply { + bytes result = 10; + Error error = 11; + } +} + +message ServiceCast { + + message MetricData { + bytes route_key = 1; + bytes metric = 2; + } + + message TopicEvent { + string topic = 1; + bytes content = 2; + } + + oneof body { + TopicEvent topic_event = 10; + MetricData metric_data = 11; + } + +} diff --git a/apps/efka/rebar.config b/apps/efka/rebar.config new file mode 100644 index 0000000..599df9c --- /dev/null +++ b/apps/efka/rebar.config @@ -0,0 +1,23 @@ +{erl_opts, [{i, "include"}]}. + +{gpb_opts, [ + {i, "proto"}, + {f, ["service.proto"]}, + recursive, + {module_name_prefix, "efka_"}, + {module_name_suffix, "_pb"}, + {o_erl, "src/protobuf"}, + {o_hrl, "include"}, + include_as_lib, + {strings_as_binaries, true}, + type_specs, + report, + verbose +]}. + +{provider_hooks, [ + {pre, [ + {compile, {protobuf, compile}}, + {clean, {protobuf, clean}} + ]} +]}. diff --git a/apps/efka/src/efka.app.src b/apps/efka/src/efka.app.src index beb2604..16ed106 100644 --- a/apps/efka/src/efka.app.src +++ b/apps/efka/src/efka.app.src @@ -6,20 +6,14 @@ {applications, [ sync, - jiffy, - %gpb, - mnesia, - parse_trans, - lager, + docker, + cowboy, + ranch, crypto, + cowlib, inets, ssl, public_key, - - %erts, - %runtime_tools, - %observer, - kernel, stdlib ]}, diff --git a/apps/efka/src/efka_agent.erl b/apps/efka/src/efka_agent.erl deleted file mode 100644 index c839dc1..0000000 --- a/apps/efka/src/efka_agent.erl +++ /dev/null @@ -1,435 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 21. 5月 2025 18:38 -%%%------------------------------------------------------------------- --module(efka_agent). --author("anlicheng"). --include("message_pb.hrl"). --include("efka.hrl"). --include("efka_tables.hrl"). - --behaviour(gen_statem). - -%% API --export([start_link/0]). --export([metric_data/3, event/3, ping/13, request_service_config/2, await_reply/2]). - -%% gen_statem callbacks --export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]). - --define(SERVER, ?MODULE). - -%% 标记当前agent的状态,只有在 activated 状态下才可以正常的发送数据 --define(STATE_DENIED, denied). --define(STATE_CONNECTING, connecting). --define(STATE_AUTH, auth). -%% 不能推送消息到服务,但是可以接受服务器的部分指令 --define(STATE_RESTRICTED, restricted). -%% 激活状态下 --define(STATE_ACTIVATED, activated). - --record(state, { - transport_pid :: undefined | pid(), - transport_ref :: undefined | reference(), - %% 服务器端推送的消息的未确认列表, 映射关系 #{Ref => PacketId} - push_inflight = #{}, - %% 发送的请求的未确认列表, 映射关系 #{Ref => ReceiverPid} - request_inflight = #{} -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - -%% 发送数据 --spec metric_data(ServiceId :: binary(), DeviceUUID::binary(), LineProtocolData :: binary()) -> no_return(). -metric_data(ServiceId, DeviceUUID, LineProtocolData) when is_binary(ServiceId), is_binary(DeviceUUID), is_binary(LineProtocolData) -> - gen_statem:cast(?SERVER, {metric_data, ServiceId, DeviceUUID, LineProtocolData}). - --spec event(ServiceId :: binary(), EventType :: integer(), Params :: binary()) -> no_return(). -event(ServiceId, EventType, Params) when is_binary(ServiceId), is_integer(EventType), is_binary(Params) -> - gen_statem:cast(?SERVER, {event, ServiceId, EventType, Params}). - -ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) -> - gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}). - -%% 请求微服务的配置 --spec request_service_config(ReceiverPid :: pid(), ServiceId :: binary()) -> {ok, Ref :: reference()} | {error, Reason :: term()}. -request_service_config(ReceiverPid, ServiceId) when is_binary(ServiceId) -> - gen_statem:call(?SERVER, {request_service_config, ReceiverPid, ServiceId}). - -%% 等待消息的回复 --spec await_reply(Ref :: reference(), Timeout :: timeout()) -> {ok, Reply :: binary()} | {error, timeout}. -await_reply(Ref, Timeout) when is_reference(Ref), is_integer(Timeout) -> - receive - {request_reply, Ref, ReplyBin} -> - {ok, ReplyBin} - after Timeout -> - {error, timeout} - end. - -%% @doc Creates a gen_statem process which calls Module:init/1 to -%% initialize. To ensure a synchronized start-up procedure, this -%% function does not return until Module:init/1 has returned. -start_link() -> - gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []). - -%%%=================================================================== -%%% gen_statem callbacks -%%%=================================================================== - -%% @private -%% @doc Whenever a gen_statem is started using gen_statem:start/[3,4] or -%% gen_statem:start_link/[3,4], this function is called by the new -%% process to initialize. -init([]) -> - erlang:start_timer(0, self(), create_transport), - {ok, ?STATE_DENIED, #state{}}. - -%% @private -%% @doc This function is called by a gen_statem when it needs to find out -%% the callback mode of the callback module. -callback_mode() -> - handle_event_function. - -%% @private -%% @doc If callback_mode is handle_event_function, then whenever a -%% gen_statem receives an event from call/2, cast/2, or as a normal -%% process message, this function is called. -handle_event({call, From}, {request_service_config, ReceiverPid, ServiceId}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid, request_inflight = RequestInflight}) -> - Ref = efka_transport:request(TransportPid, ?METHOD_REQUEST_SERVICE_CONFIG, ServiceId), - {keep_state, State#state{request_inflight = maps:put(Ref, ReceiverPid, RequestInflight)}, [{reply, From, {ok, Ref}}]}; - -handle_event({call, From}, {request_service_config, _ReceiverPid, _ServiceId}, _, State) -> - {keep_state, State, [{reply, From, {error, <<"transport is not alive">>}}]}; - -%% 异步发送数据, 连接存在时候直接发送;否则缓存到mnesia -handle_event(cast, {metric_data, ServiceId, DeviceUUID, LineProtocolData}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) -> - Packet = message_pb:encode_msg(#data{ - service_id = ServiceId, - device_uuid = DeviceUUID, - metric = LineProtocolData - }), - efka_transport:send(TransportPid, ?METHOD_DATA, Packet), - {keep_state, State}; - -handle_event(cast, {metric_data, ServiceId, DeviceUUID, LineProtocolData}, _, State) -> - Packet = message_pb:encode_msg(#data{ - service_id = ServiceId, - device_uuid = DeviceUUID, - metric = LineProtocolData - }), - ok = cache_model:insert(?METHOD_DATA, Packet), - {keep_state, State}; - -%% 异步发送事件 -handle_event(cast, {event, ServiceId, EventType, Params}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) -> - EventPacket = message_pb:encode_msg(#event{ - service_id = ServiceId, - event_type = EventType, - params = Params - }), - efka_transport:send(TransportPid, ?METHOD_EVENT, EventPacket), - {keep_state, State}; -handle_event(cast, {event, ServiceId, EventType, Params}, ?STATE_ACTIVATED, State) -> - EventPacket = message_pb:encode_msg(#event{ - service_id = ServiceId, - event_type = EventType, - params = Params - }), - ok = cache_model:insert(?METHOD_EVENT, EventPacket), - {keep_state, State}; - -handle_event(cast, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}, ?STATE_ACTIVATED, - State = #state{transport_pid = TransportPid}) -> - - Ping = message_pb:encode_msg(#ping{ - adcode = AdCode, - boot_time = BootTime, - province = Province, - city = City, - efka_version = EfkaVersion, - kernel_arch = KernelArch, - ips = Ips, - cpu_core = CpuCore, - cpu_load = CpuLoad, - cpu_temperature = CpuTemperature, - disk = Disk, - memory = Memory, - interfaces = Interfaces - }), - efka_transport:send(TransportPid, ?METHOD_PING, Ping), - {keep_state, State}; - -%% 异步建立到服务器的连接 -handle_event(info, {timeout, _, create_transport}, ?STATE_DENIED, State) -> - {ok, Props} = application:get_env(efka, tls_server), - Host = proplists:get_value(host, Props), - Port = proplists:get_value(port, Props), - {ok, {TransportPid, TransportRef}} = efka_transport:start_monitor(self(), Host, Port), - efka_transport:connect(TransportPid), - - {next_state, ?STATE_CONNECTING, State#state{transport_pid = TransportPid, transport_ref = TransportRef}}; - -handle_event(info, {connect_reply, Reply}, ?STATE_CONNECTING, State = #state{transport_pid = TransportPid}) -> - case Reply of - ok -> - AuthBin = auth_request(), - efka_transport:auth_request(TransportPid, AuthBin), - {next_state, ?STATE_AUTH, State}; - {error, Reason} -> - lager:debug("[efka_agent] connect failed, error: ~p, pid: ~p", [Reason, TransportPid]), - efka_transport:stop(TransportPid), - {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}} - end; - -handle_event(info, {auth_reply, Reply}, ?STATE_AUTH, State = #state{transport_pid = TransportPid}) -> - case Reply of - {ok, ReplyBin} -> - #auth_reply{code = Code, message = Message} = message_pb:decode_msg(ReplyBin, auth_reply), - case Code of - 0 -> - lager:debug("[efka_agent] auth success, message: ~p", [Message]), - {next_state, ?STATE_ACTIVATED, State, [{next_event, info, flush_cache}]}; - 1 -> - %% 主机在后台的授权未通过;此时agent不能推送数据给云端服务器,但是云端服务器可以推送命令给agent - %% socket的连接状态需要维持 - lager:debug("[efka_agent] auth denied, message: ~p", [Message]), - {next_state, ?STATE_RESTRICTED, State}; - 2 -> - % 其他类型的错误,需要间隔时间重试 - lager:debug("[efka_agent] auth failed, message: ~p", [Message]), - efka_transport:stop(TransportPid), - {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}}; - _ -> - % 其他类型的错误,需要间隔时间重试 - lager:debug("[efka_agent] auth failed, invalid message"), - efka_transport:stop(TransportPid), - {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}} - end; - {error, Reason} -> - lager:debug("[efka_agent] auth_request failed, error: ~p", [Reason]), - efka_transport:stop(TransportPid), - {next_state, ?STATE_DENIED, State#state{transport_pid = undefined}} - end; - -%% 将缓存中的数据推送到服务器端 -handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) -> - case cache_model:fetch_next() of - {ok, #cache{id = Id, method = Method, data = Packet}} -> - efka_transport:send(TransportPid, Method, Packet), - cache_model:delete(Id), - {keep_state, State, [{next_event, info, flush_cache}]}; - error -> - {keep_state, State} - end; -handle_event(info, flush_cache, _, State) -> - {keep_state, State}; - -%% 云端服务器推送了消息 -%% 激活消息 - -%% 微服务部署 -handle_event(info, {server_async_call, PacketId, <>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) -> - #deploy{task_id = TaskId, service_id = ServiceId, tar_url = TarUrl} = message_pb:decode_msg(DeployBin, deploy), - - %% 短暂的等待,efka_inetd收到消息后就立即返回了 - Reply = case efka_inetd:deploy(TaskId, ServiceId, TarUrl) of - ok -> - #async_call_reply{code = 1, result = <<"ok">>}; - {error, Reason} when is_binary(Reason) -> - #async_call_reply{code = 0, message = Reason} - end, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - - {keep_state, State}; - -%% 启动微服务 -handle_event(info, {server_async_call, PacketId, <>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) -> - %% 短暂的等待,efka_inetd收到消息后就立即返回了 - Reply = case efka_inetd:start_service(ServiceId) of - ok -> - #async_call_reply{code = 1, result = <<"ok">>}; - {error, Reason} when is_binary(Reason) -> - #async_call_reply{code = 0, message = Reason} - end, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - - {keep_state, State}; - -%% 停止微服务 -handle_event(info, {server_async_call, PacketId, <>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) -> - %% 短暂的等待,efka_inetd收到消息后就立即返回了 - Reply = case efka_inetd:stop_service(ServiceId) of - ok -> - #async_call_reply{code = 1, result = <<"ok">>}; - {error, Reason} when is_binary(Reason) -> - #async_call_reply{code = 0, message = Reason} - end, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - - {keep_state, State}; - -%% config.json配置信息 -handle_event(info, {server_async_call, PacketId, <>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid, push_inflight = PushInflight}) -> - #push_service_config{service_id = ServiceId, config_json = ConfigJson, timeout = Timeout} = message_pb:decode_msg(ConfigBin, push_service_config), - - case efka_service:get_pid(ServiceId) of - undefined -> - Reply = #async_call_reply{code = 0, message = <<"service not run">>}, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - {keep_state, State}; - ServicePid when is_pid(ServicePid) -> - Ref = make_ref(), - %% 将配置文件推送到对应的微服务 - efka_service:push_config(ServicePid, Ref, ConfigJson), - %% 处理超时逻辑 - erlang:start_timer(Timeout, self(), {request_timeout, Ref}), - - {keep_state, State#state{push_inflight = maps:put(Ref, PacketId, PushInflight)}} - end; - -%% 收到需要回复的指令 -handle_event(info, {server_async_call, PacketId, <>}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) -> - #invoke{service_id = ServiceId, payload = Payload, timeout = Timeout} = message_pb:decode_msg(InvokeBin, invoke), - %% 消息发送到订阅系统 - case efka_service:get_pid(ServiceId) of - undefined -> - Reply = #async_call_reply{code = 0, message = <<"micro_service not run">>}, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - - {keep_state, State}; - ServicePid when is_pid(ServicePid) -> - Ref = make_ref(), - efka_service:invoke(ServicePid, Ref, Payload), - %% 处理超时逻辑 - erlang:start_timer(Timeout, self(), {request_timeout, Ref}), - - {keep_state, State#state{push_inflight = maps:put(Ref, PacketId, PushInflight)}} - end; - -%% 处理task_log -handle_event(info, {server_async_call, PacketId, <>}, ?STATE_ACTIVATED, State = #state{transport_pid = TransportPid}) -> - #fetch_task_log{task_id = TaskId} = message_pb:decode_msg(TaskLogBin, fetch_task_log), - lager:debug("[efka_agent] get task_log request: ~p", [TaskId]), - {ok, Logs} = efka_inetd_task_log:get_logs(TaskId), - Reply = case length(Logs) > 0 of - true -> - Result = iolist_to_binary(jiffy:encode(Logs, [force_utf8])), - #async_call_reply{code = 1, result = Result}; - false -> - #async_call_reply{code = 1, result = <<"[]">>} - end, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - - {keep_state, State}; - -%% 处理命令 -handle_event(info, {server_command, ?COMMAND_AUTH, <>}, StateName, State = #state{transport_pid = TransportPid}) -> - case {Auth, StateName} of - {1, ?STATE_ACTIVATED} -> - {keep_state, State}; - {1, ?STATE_DENIED} -> - %% 重新激活, 需要重新校验 - AuthRequestBin = auth_request(), - efka_transport:auth_request(TransportPid, AuthRequestBin), - {next_state, ?STATE_AUTH, State}; - {0, _} -> - %% 这个时候的主机应该是受限制的状态,不允许发送消息;但是能够接受服务器推送的消息 - {next_state, ?STATE_RESTRICTED, State} - end; - -%% 处理Pub/Sub机制 -handle_event(info, {server_pub, Topic, Content}, ?STATE_ACTIVATED, State) -> - lager:debug("[efka_agent] get pub topic: ~p, content: ~p", [Topic, Content]), - %% 消息发送到订阅系统 - efka_subscription:publish(Topic, Content), - {keep_state, State}; - -%% 收到来自efka_service的回复 -handle_event(info, {service_reply, Ref, EmsReply}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) -> - case maps:take(Ref, PushInflight) of - error -> - {keep_state, State}; - {PacketId, NPushInflight} -> - Reply = case EmsReply of - {ok, Result} -> - #async_call_reply{code = 1, result = Result}; - {error, Reason} -> - #async_call_reply{code = 0, message = Reason} - end, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - - {keep_state, State#state{push_inflight = NPushInflight}} - end; - -%% 收到来自服务器端的回复 -handle_event(info, {server_reply, Ref, ReplyBin}, ?STATE_ACTIVATED, State = #state{request_inflight = RequestInflight}) -> - case maps:take(Ref, RequestInflight) of - error -> - {keep_state, State}; - {ReceiverPid, NRequestInflight} -> - is_process_alive(ReceiverPid) andalso erlang:send(ReceiverPid, {request_reply, Ref, ReplyBin}), - {keep_state, State#state{push_inflight = NRequestInflight}} - end; - -%% todo 请求超时逻辑处理 -handle_event(info, {timeout, _, {request_timeout, Ref}}, ?STATE_ACTIVATED, State = #state{push_inflight = PushInflight, transport_pid = TransportPid}) -> - case maps:take(Ref, PushInflight) of - error -> - {keep_state, State}; - {PacketId, NPushInflight} -> - Reply = #async_call_reply{code = 0, message = <<"reqeust timeout">>, result = <<>>}, - efka_transport:async_call_reply(TransportPid, PacketId, message_pb:encode_msg(Reply)), - - {keep_state, State#state{push_inflight = NPushInflight}} - end; - -%% transport进程退出 -handle_event(info, {'DOWN', MRef, process, TransportPid, Reason}, _, State = #state{transport_ref = MRef}) -> - lager:debug("[efka_agent] transport pid: ~p, exit with reason: ~p", [TransportPid, Reason]), - erlang:start_timer(5000, self(), create_transport), - {next_state, ?STATE_DENIED, State#state{transport_pid = undefined, transport_ref = undefined}}. - -%% @private -%% @doc This function is called by a gen_statem when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_statem terminates with -%% Reason. The return value is ignored. -terminate(_Reason, _StateName, _State = #state{transport_pid = TransportPid}) -> - case is_pid(TransportPid) andalso is_process_alive(TransportPid) of - true -> - efka_transport:stop(TransportPid); - false -> - ok - end, - ok. - -%% @private -%% @doc Convert process state when code is changed -code_change(_OldVsn, StateName, State = #state{}, _Extra) -> - {ok, StateName, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== - --spec auth_request() -> binary(). -auth_request() -> - {ok, AuthInfo} = application:get_env(efka, auth), - UUID = proplists:get_value(uuid, AuthInfo), - Username = proplists:get_value(username, AuthInfo), - Salt = proplists:get_value(salt, AuthInfo), - Token = proplists:get_value(token, AuthInfo), - - message_pb:encode_msg(#auth_request{ - uuid = unicode:characters_to_binary(UUID), - username = unicode:characters_to_binary(Username), - salt = unicode:characters_to_binary(Salt), - token = unicode:characters_to_binary(Token), - timestamp = efka_util:timestamp() - }). \ No newline at end of file diff --git a/apps/efka/src/efka_app.erl b/apps/efka/src/efka_app.erl index 9fbea81..cc6f27d 100644 --- a/apps/efka/src/efka_app.erl +++ b/apps/efka/src/efka_app.erl @@ -9,46 +9,38 @@ -export([start/2, stop/1]). +-spec start(term(), term()) -> {ok, pid()} | {error, term()}. start(_StartType, _StartArgs) -> io:setopts([{encoding, unicode}]), - %% 启动mnesia数据库 - start_mnesia(), %% 加速内存的回收 erlang:system_flag(fullsweep_after, 16), + start_websocket_server(), + efka_sup:start_link(). +-spec stop(term()) -> ok. stop(_State) -> ok. -%% internal functions +%% 微服务和efka之间通过websocket协议通讯 +-spec start_websocket_server() -> ok. +start_websocket_server() -> + {ok, Props} = application:get_env(efka, websocket_server), + Acceptors = proplists:get_value(acceptors, Props, 50), + MaxConnections = proplists:get_value(max_connections, Props, 10240), + Backlog = proplists:get_value(backlog, Props, 1024), + Port = proplists:get_value(port, Props), -%% 启动内存数据库 -start_mnesia() -> - %% 启动数据库 - ensure_mnesia_schema(), - ok = mnesia:start(), - Tables = mnesia:system_info(tables), - lager:debug("[efka_app] tables: ~p", [Tables]), - %% 创建数据库表 - not lists:member(id_generator, Tables) andalso id_generator_model:create_table(), - not lists:member(service, Tables) andalso service_model:create_table(), - not lists:member(cache, Tables) andalso cache_model:create_table(), - not lists:member(task_log, Tables) andalso task_log_model:create_table(), - ok. + Dispatcher = cowboy_router:compile([ + {'_', [{"/ws", efka_service_channel, []}]} + ]), --spec ensure_mnesia_schema() -> any(). -ensure_mnesia_schema() -> - case mnesia:system_info(use_dir) of - true -> - lager:debug("[efka_app] mnesia schema exists"), - ok; - false -> - mnesia:stop(), - case mnesia:create_schema([node()]) of - ok -> ok; - {error, {_, {already_exists, _}}} -> ok; - Error -> - lager:debug("[iot_app] create mnesia schema failed with error: ~p", [Error]), - throw({init_schema, Error}) - end - end. \ No newline at end of file + TransOpts = [ + {port, Port}, + {num_acceptors, Acceptors}, + {backlog, Backlog}, + {max_connections, MaxConnections} + ], + {ok, Pid} = cowboy:start_clear(ws_listener, TransOpts, #{env => #{dispatch => Dispatcher}}), + + logger:debug("[efka_app] websocket server start at: ~p, pid is: ~p", [Port, Pid]). diff --git a/apps/efka/src/efka_inetd.erl b/apps/efka/src/efka_inetd.erl deleted file mode 100644 index b9b0dfb..0000000 --- a/apps/efka/src/efka_inetd.erl +++ /dev/null @@ -1,212 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% 微服务守护进程 -%%% 1. 负责微服务的下载, 版本的管理 -%%% 2. 目录管理等 -%%% @end -%%% Created : 19. 4月 2025 14:55 -%%%------------------------------------------------------------------- --module(efka_inetd). --author("anlicheng"). --include("efka_tables.hrl"). --include("message_pb.hrl"). - --behaviour(gen_server). - -%% API --export([start_link/0]). --export([deploy/3, start_service/1, stop_service/1]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). - --define(SERVER, ?MODULE). - --record(state, { - root_dir :: string(), - %% 建立任务到ref之间的映射, #{TaskPid => {TaskId, ServiceId}} - task_map = #{} -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - --spec deploy(TaskId :: integer(), ServerId :: binary(), TarUrl :: binary()) -> ok | {error, Reason :: binary()}. -deploy(TaskId, ServerId, TarUrl) when is_integer(TaskId), is_binary(ServerId), is_binary(TarUrl) -> - gen_server:call(?SERVER, {deploy, TaskId, ServerId, TarUrl}). - --spec start_service(ServiceId :: binary()) -> ok | {error, Reason :: term()}. -start_service(ServiceId) when is_binary(ServiceId) -> - gen_server:call(?SERVER, {start_service, ServiceId}). - --spec stop_service(ServiceId :: binary()) -> ok | {error, Reason :: term()}. -stop_service(ServiceId) when is_binary(ServiceId) -> - gen_server:call(?SERVER, {stop_service, ServiceId}). - -%% @doc Spawns the server and registers the local name (unique) --spec(start_link() -> - {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). -start_link() -> - gen_server:start_link({local, ?SERVER}, ?MODULE, [], []). - -%%%=================================================================== -%%% gen_server callbacks -%%%=================================================================== - -%% @private -%% @doc Initializes the server --spec(init(Args :: term()) -> - {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | - {stop, Reason :: term()} | ignore). -init([]) -> - erlang:process_flag(trap_exit, true), - {ok, RootDir} = application:get_env(efka, root_dir), - {ok, #state{root_dir = RootDir}}. - -%% @private -%% @doc Handling call messages --spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, - State :: #state{}) -> - {reply, Reply :: term(), NewState :: #state{}} | - {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_call({deploy, TaskId, ServiceId, TarUrl}, _From, State = #state{root_dir = RootDir, task_map = TaskMap}) -> - %% 创建目录 - {ok, ServiceRootDir} = ensure_dirs(RootDir, ServiceId), - - ServicePid = efka_service:get_pid(ServiceId), - case is_pid(ServicePid) of - true -> - {reply, {error, <<"the service is running, stop first">>}, State}; - false -> - case check_download_url(TarUrl) of - ok -> - {ok, TaskPid} = efka_inetd_task:start_link(TaskId, ServiceRootDir, ServiceId, TarUrl), - efka_inetd_task:deploy(TaskPid), - lager:debug("[efka_inetd] start task_id: ~p, tar_url: ~p", [TaskId, TarUrl]), - - {reply, ok, State#state{task_map = maps:put(TaskPid, {TaskId, ServiceId}, TaskMap)}}; - {error, Reason} -> - lager:debug("[efka_inetd] check_download_url: ~p, get error: ~p", [TarUrl, Reason]), - {reply, {error, <<"download url error">>}, State} - end - end; - -%% 启动服务: 当前服务如果正常运行,则不允许重启 -handle_call({start_service, ServiceId}, _From, State) -> - case efka_service:get_pid(ServiceId) of - undefined -> - case efka_service_sup:start_service(ServiceId) of - {ok, _} -> - %% 更新数据库状态, 状态是为了保证下次efka重启的时候,服务能够启动 - ok = service_model:change_status(ServiceId, 1), - {reply, ok, State}; - {error, Reason} -> - {reply, {error, Reason}, State} - end; - ServicePid when is_pid(ServicePid) -> - {reply, {error, <<"service is running">>}, State} - end; - -%% 停止服务, 主动停止的时候会改变服务配置的status字段 -handle_call({stop_service, ServiceId}, _From, State = #state{}) -> - case efka_service:get_pid(ServiceId) of - undefined -> - {reply, {error, <<"service not running">>}, State}; - ServicePid when is_pid(ServicePid) -> - efka_service_sup:stop_service(ServiceId), - %% 主动停止的服务,需要更新数据库状态, 状态是为了保证下次efka重启的时候,不自动启动服务 - ok = service_model:change_status(ServiceId, 0), - {reply, ok, State} - end; - -handle_call(_Request, _From, State = #state{}) -> - {reply, ok, State}. - -%% @private -%% @doc Handling cast messages --spec(handle_cast(Request :: term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_cast(_Request, State = #state{}) -> - {noreply, State}. - -%% @private -%% @doc Handling all non call/cast messages --spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_info({'EXIT', TaskPid, Reason}, State = #state{task_map = TaskMap}) -> - case maps:take(TaskPid, TaskMap) of - error -> - {noreply, State}; - {{TaskId, ServiceId}, NTaskMap} -> - case Reason of - normal -> - lager:debug("[efka_inetd] service_id: ~p, task_pid: ~p, exit normal", [ServiceId, TaskPid]), - efka_inetd_task_log:flush(TaskId); - Error -> - lager:notice("[efka_inetd] service_id: ~p, task_pid: ~p, exit with error: ~p", [ServiceId, TaskPid, Error]), - efka_inetd_task_log:stash(TaskId, <<"task aborted">>), - efka_inetd_task_log:flush(TaskId) - end, - {noreply, State#state{task_map = NTaskMap}} - end; - -handle_info(_Info, State = #state{}) -> - {noreply, State}. - -%% @private -%% @doc This function is called by a gen_server when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_server terminates -%% with Reason. The return value is ignored. --spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), - State :: #state{}) -> term()). -terminate(_Reason, _State = #state{}) -> - ok. - -%% @private -%% @doc Convert process state when code is changed --spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, - Extra :: term()) -> - {ok, NewState :: #state{}} | {error, Reason :: term()}). -code_change(_OldVsn, State = #state{}, _Extra) -> - {ok, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== - --spec ensure_dirs(RootDir :: string(), ServerId :: binary()) -> {ok, ServerRootDir :: string()}. -ensure_dirs(RootDir, ServerId) when is_list(RootDir), is_binary(ServerId) -> - %% 根目录 - ServiceRootDir = RootDir ++ "/" ++ binary_to_list(ServerId) ++ "/", - ok = filelib:ensure_dir(ServiceRootDir), - {ok, ServiceRootDir}. - -%% 通过head请求先判定下载地址是否正确 --spec check_download_url(Url :: string() | binary()) -> ok | {error, Reason :: term()}. -check_download_url(Url) when is_binary(Url) -> - check_download_url(binary_to_list(Url)); -check_download_url(Url) when is_list(Url) -> - SslOpts = [ - {ssl, [ - % 完全禁用证书验证 - {verify, verify_none} - ]} - ], - case httpc:request(head, {Url, []}, SslOpts, [{sync, true}]) of - {ok, {{_, 200, "OK"}, _Headers, _}} -> - ok; - {error, Reason} -> - {error, Reason} - end. \ No newline at end of file diff --git a/apps/efka/src/efka_inetd_task.erl b/apps/efka/src/efka_inetd_task.erl deleted file mode 100644 index 3bb455a..0000000 --- a/apps/efka/src/efka_inetd_task.erl +++ /dev/null @@ -1,241 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 07. 5月 2025 15:47 -%%%------------------------------------------------------------------- --module(efka_inetd_task). --author("anlicheng"). --include("efka_tables.hrl"). - --behaviour(gen_server). - -%% API --export([start_link/4]). --export([deploy/1]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). - --define(SERVER, ?MODULE). - --record(state, { - service_root_dir :: string(), - task_id :: integer(), - service_id :: binary(), - tar_url :: binary() -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - --spec deploy(Pid :: pid()) -> no_return(). -deploy(Pid) when is_pid(Pid) -> - gen_server:cast(Pid, deploy). - -%% @doc Spawns the server and registers the local name (unique) --spec(start_link(TaskId :: integer(), ServiceRootDir :: string(), ServiceId :: binary(), TarUrl :: binary()) -> - {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). -start_link(TaskId, ServiceRootDir, ServiceId, TarUrl) when is_integer(TaskId), is_list(ServiceRootDir), is_binary(ServiceId), is_binary(TarUrl) -> - gen_server:start_link(?MODULE, [TaskId, ServiceRootDir, ServiceId, TarUrl], []). - -%%%=================================================================== -%%% gen_server callbacks -%%%=================================================================== - -%% @private -%% @doc Initializes the server --spec(init(Args :: term()) -> - {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | - {stop, Reason :: term()} | ignore). -init([TaskId, ServiceRootDir, ServiceId, TarUrl]) -> - {ok, #state{task_id = TaskId, service_root_dir = ServiceRootDir, service_id = ServiceId, tar_url = TarUrl}}. - -%% @private -%% @doc Handling call messages --spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, - State :: #state{}) -> - {reply, Reply :: term(), NewState :: #state{}} | - {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_call(_Request, _From, State = #state{}) -> - {reply, ok, State}. - -%% @private -%% @doc Handling cast messages --spec(handle_cast(Request :: term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_cast(deploy, State = #state{task_id = TaskId, service_root_dir = ServiceRootDir, service_id = ServiceId, tar_url = TarUrl}) -> - do_deploy(TaskId, ServiceRootDir, ServiceId, TarUrl), - {stop, normal, State}; -handle_cast(_Request, State) -> - {stop, normal, State}. - -%% @private -%% @doc Handling all non call/cast messages --spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_info(_Info, State = #state{}) -> - {noreply, State}. - -%% @private -%% @doc This function is called by a gen_server when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_server terminates -%% with Reason. The return value is ignored. --spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), - State :: #state{}) -> term()). -terminate(_Reason, _State = #state{}) -> - ok. - -%% @private -%% @doc Convert process state when code is changed --spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, - Extra :: term()) -> - {ok, NewState :: #state{}} | {error, Reason :: term()}). -code_change(_OldVsn, State = #state{}, _Extra) -> - {ok, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== - --spec do_deploy(TaskId :: integer(), ServiceRootDir :: string(), ServiceId :: binary(), TarUrl :: binary()) -> no_return(). -do_deploy(TaskId, ServiceRootDir, ServiceId, TarUrl) when is_integer(TaskId), is_list(ServiceRootDir), is_binary(ServiceId), is_binary(TarUrl) -> - case download(binary_to_list(TarUrl), ServiceRootDir) of - {ok, TarFile, CostTs} -> - Log = io_lib:format("download: ~p completed, cost time: ~p(ms)", [binary_to_list(TarUrl), CostTs]), - efka_inetd_task_log:stash(TaskId, list_to_binary(Log)), - - %% 创建工作目录 - WorkDir = ServiceRootDir ++ "/work_dir/", - case filelib:ensure_dir(WorkDir) of - ok -> - %% 清理目录下的文件 - catch delete_directory(WorkDir), - case tar_extract(TarFile, WorkDir) of - ok -> - %% 更新数据 - ok = service_model:insert(#service{ - service_id = ServiceId, - tar_url = TarUrl, - %% 工作目录 - root_dir = ServiceRootDir, - config_json = <<"">>, - %% 状态: 0: 停止, 1: 运行中 - status = 0 - }), - efka_inetd_task_log:stash(TaskId, <<"deploy success">>); - {error, Reason} -> - TarLog = io_lib:format("tar decompression: ~p, error: ~p", [filename:basename(TarFile), Reason]), - efka_inetd_task_log:stash(TaskId, list_to_binary(TarLog)) - end; - {error, Reason} -> - DownloadLog = io_lib:format("make work_dir error: ~p", [Reason]), - efka_inetd_task_log:stash(TaskId, list_to_binary(DownloadLog)) - end; - {error, Reason} -> - DownloadLog = io_lib:format("download: ~p, error: ~p", [binary_to_list(TarUrl), Reason]), - efka_inetd_task_log:stash(TaskId, list_to_binary(DownloadLog)) - end. - -%% 递归删除目录下的问题 --spec delete_directory(string()) -> ok | {error, term()}. -delete_directory(Dir) when is_list(Dir) -> - % 递归删除目录内容 - case file:list_dir(Dir) of - {ok, Files} -> - lists:foreach(fun(File) -> - FullPath = filename:join(Dir, File), - case filelib:is_dir(FullPath) of - true -> - delete_directory(FullPath); - false -> - file:delete(FullPath) - end - end, Files), - % 删除空目录 - file:del_dir(Dir); - {error, enoent} -> - ok; - {error, Reason} -> - {error, Reason} - end. - -%% 解压文件到指定目录 --spec tar_extract(string(), string()) -> ok | {error, term()}. -tar_extract(TarFile, TargetDir) when is_list(TarFile), is_list(TargetDir) -> - %% 判断文件的后缀名来判断, options: verbose - erl_tar:extract(TarFile, [compressed, {cwd, TargetDir}]). - -%% 下载文件 --spec download(Url :: string(), TargetDir :: string()) -> - {ok, TarFile :: string(), CostTs :: integer()} | {error, Reason :: term()}. -download(Url, TargetDir) when is_list(Url), is_list(TargetDir) -> - SslOpts = [ - {ssl, [ - % 完全禁用证书验证 - {verify, verify_none} - ]} - ], - - TargetFile = get_filename_from_url(Url), - FullFilename = TargetDir ++ TargetFile, - - StartTs = os:timestamp(), - case httpc:request(get, {Url, []}, SslOpts, [{sync, false}, {stream, self}]) of - {ok, RequestId} -> - case receive_data(RequestId, FullFilename) of - ok -> - EndTs = os:timestamp(), - %% 计算操作的时间,单位为毫秒 - CostMs = timer:now_diff(EndTs, StartTs) div 1000, - {ok, FullFilename, CostMs}; - {error, Reason} -> - %% 出错需要删除掉文件 - file:delete(FullFilename), - {error, Reason} - end; - {error, Reason} -> - {error, Reason} - end. - -%% 处理头部信息, 解析可能的文件名 -receive_data(RequestId, FullFilename) -> - receive - {http, {RequestId, stream_start, _Headers}} -> - {ok, File} = file:open(FullFilename, [write, binary]), - receive_data0(RequestId, File); - {http, {RequestId, {{_, 404, Status}, _Headers, Body}}} -> - lager:debug("[efka_downloader] http_status: ~p, body: ~p", [Status, Body]), - {error, Status} - end. -%% 接受文件数据 -receive_data0(RequestId, File) -> - receive - {http, {RequestId, {error, Reason}}} -> - ok = file:close(File), - {error, Reason}; - {http, {RequestId, stream_end, _Headers}} -> - ok = file:close(File), - ok; - {http, {RequestId, stream, Data}} -> - file:write(File, Data), - receive_data0(RequestId, File) - end. - --spec get_filename_from_url(Url :: string()) -> string(). -get_filename_from_url(Url) when is_list(Url) -> - URIMap = uri_string:parse(Url), - Path = maps:get(path, URIMap), - filename:basename(Path). \ No newline at end of file diff --git a/apps/efka/src/efka_inetd_task_log.erl b/apps/efka/src/efka_inetd_task_log.erl deleted file mode 100644 index 9ce361b..0000000 --- a/apps/efka/src/efka_inetd_task_log.erl +++ /dev/null @@ -1,136 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 09. 5月 2025 16:45 -%%%------------------------------------------------------------------- --module(efka_inetd_task_log). --author("anlicheng"). - --behaviour(gen_server). - -%% API --export([start_link/0]). --export([stash/2, flush/1, get_logs/1]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). - --define(SERVER, ?MODULE). - --record(state, { - %% #{task_id => queue:new()} - pending_map = #{} -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - --spec stash(TaskId :: integer(), Items :: binary() | [binary()]) -> no_return(). -stash(TaskId, Log) when is_integer(TaskId), is_binary(Log) -> - stash(TaskId, [Log]); -stash(TaskId, Items) when is_integer(TaskId), is_list(Items) -> - {{Y, M, D}, {H, I, S}} = calendar:local_time(), - TimePrefix = iolist_to_binary(io_lib:format("[~b-~2..0b-~2..0b ~2..0b:~2..0b:~2..0b]", [Y, M, D, H, I, S])), - Log = iolist_to_binary([TimePrefix, <<" ">>, lists:join(<<" ">>, Items)]), - gen_server:cast(?SERVER, {stash, TaskId, Log}). - --spec flush(TaskId :: integer()) -> no_return(). -flush(TaskId) when is_integer(TaskId) -> - gen_server:cast(?SERVER, {flush, TaskId}). - --spec get_logs(TaskId :: integer()) -> {ok, Logs :: list()}. -get_logs(TaskId) when is_integer(TaskId) -> - gen_server:call(?SERVER, {get_logs, TaskId}). - -%% @doc Spawns the server and registers the local name (unique) --spec(start_link() -> - {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). -start_link() -> - gen_server:start_link({local, ?SERVER}, ?MODULE, [], []). - -%%%=================================================================== -%%% gen_server callbacks -%%%=================================================================== - -%% @private -%% @doc Initializes the server --spec(init(Args :: term()) -> - {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | - {stop, Reason :: term()} | ignore). -init([]) -> - {ok, #state{}}. - -%% @private -%% @doc Handling call messages --spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, - State :: #state{}) -> - {reply, Reply :: term(), NewState :: #state{}} | - {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_call({get_logs, TaskId}, _From, State = #state{pending_map = PendingMap}) -> - case maps:find(TaskId, PendingMap) of - error -> - Logs = task_log_model:get_logs(TaskId), - {reply, {ok, Logs}, State}; - {ok, Q} -> - Logs = queue:to_list(Q), - {reply, {ok, Logs}, State} - end. - -%% @private -%% @doc Handling cast messages --spec(handle_cast(Request :: term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_cast({stash, TaskId, Log}, State = #state{pending_map = PendingMap}) -> - Q = maps:get(TaskId, PendingMap, queue:new()), - NQ = queue:in(Log, Q), - {noreply, State#state{pending_map = maps:put(TaskId, NQ, PendingMap)}}; -handle_cast({flush, TaskId}, State = #state{pending_map = PendingMap}) -> - case maps:take(TaskId, PendingMap) of - error -> - {noreply, State}; - {Q, NPendingMap} -> - Logs = queue:to_list(Q), - ok = task_log_model:insert(TaskId, Logs), - {noreply, State#state{pending_map = NPendingMap}} - end. - -%% @private -%% @doc Handling all non call/cast messages --spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_info(_Info, State = #state{}) -> - {noreply, State}. - -%% @private -%% @doc This function is called by a gen_server when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_server terminates -%% with Reason. The return value is ignored. --spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), - State :: #state{}) -> term()). -terminate(_Reason, _State = #state{}) -> - ok. - -%% @private -%% @doc Convert process state when code is changed --spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, - Extra :: term()) -> - {ok, NewState :: #state{}} | {error, Reason :: term()}). -code_change(_OldVsn, State = #state{}, _Extra) -> - {ok, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== diff --git a/apps/efka/src/efka_logger.erl b/apps/efka/src/efka_logger.erl new file mode 100644 index 0000000..74a8f4a --- /dev/null +++ b/apps/efka/src/efka_logger.erl @@ -0,0 +1,179 @@ +%%%------------------------------------------------------------------- +%%% @author aresei +%%% @copyright (C) 2023, +%%% @doc +%%% +%%% @end +%%% Created : 07. 9月 2023 17:07 +%%%------------------------------------------------------------------- +-module(efka_logger). +-author("aresei"). + +-behaviour(gen_server). + +%% API +-export([start_link/1, write/1, write_lines/1]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). + +-define(SERVER, ?MODULE). + +-record(state, { + file_name :: string(), + date :: calendar:date(), + file +}). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec write(Data :: binary()) -> ok. +write(Data) when is_binary(Data) -> + gen_server:cast(?SERVER, {write, Data}). + +-spec write_lines(Lines :: [binary()]) -> ok. +write_lines(Lines) when is_list(Lines) -> + gen_server:cast(?SERVER, {write_lines, Lines}). + +%% @doc Spawns the server and registers the local name (unique) +-spec(start_link(FileName :: string()) -> + {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). +start_link(FileName) when is_list(FileName) -> + gen_server:start_link({local, ?SERVER}, ?MODULE, [FileName], []). + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +%% @private +%% @doc Initializes the server +-spec(init(Args :: term()) -> + {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | + {stop, Reason :: term()} | ignore). +init([FileName]) -> + ensure_dir(), + FilePath = make_file(FileName), + {ok, File} = file:open(FilePath, [append, binary]), + + {ok, #state{file = File, file_name = FileName, date = get_date()}}. + +%% @private +%% @doc Handling call messages +-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, + State :: #state{}) -> + {reply, Reply :: term(), NewState :: #state{}} | + {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | + {noreply, NewState :: #state{}} | + {noreply, NewState :: #state{}, timeout() | hibernate} | + {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | + {stop, Reason :: term(), NewState :: #state{}}). +handle_call(_Request, _From, State = #state{}) -> + {reply, ok, State}. + +%% @private +%% @doc Handling cast messages +-spec(handle_cast(Request :: term(), State :: #state{}) -> + {noreply, NewState :: #state{}} | + {noreply, NewState :: #state{}, timeout() | hibernate} | + {stop, Reason :: term(), NewState :: #state{}}). +handle_cast({write, Data}, State = #state{file = OldFile, file_name = FileName, date = Date}) -> + Line = <<(time_prefix())/binary, " ", (format(Data))/binary, $\n>>, + case maybe_new_file(Date) of + true -> + file:close(OldFile), + + FilePath = make_file(FileName), + {ok, File} = file:open(FilePath, [append, binary]), + ok = file:write(File, Line), + {noreply, State#state{file = File, date = get_date()}}; + false -> + ok = file:write(OldFile, Line), + {noreply, State} + end; +handle_cast({write_lines, Lines}, State = #state{file = OldFile, file_name = FileName, date = Date}) -> + Data = iolist_to_binary(lists:join(<<$\n>>, Lines)), + case maybe_new_file(Date) of + true -> + file:close(OldFile), + + FilePath = make_file(FileName), + {ok, File} = file:open(FilePath, [append, binary]), + ok = file:write(File, Data), + {noreply, State#state{file = File, date = get_date()}}; + false -> + ok = file:write(OldFile, Data), + {noreply, State} + end. + +%% @private +%% @doc Handling all non call/cast messages +-spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> + {noreply, NewState :: #state{}} | + {noreply, NewState :: #state{}, timeout() | hibernate} | + {stop, Reason :: term(), NewState :: #state{}}). +handle_info(_Info, State = #state{}) -> + {noreply, State}. + +%% @private +%% @doc This function is called by a gen_server when it is about to +%% terminate. It should be the opposite of Module:init/1 and do any +%% necessary cleaning up. When it returns, the gen_server terminates +%% with Reason. The return value is ignored. +-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), + State :: #state{}) -> term()). +terminate(_Reason, _State = #state{}) -> + ok. + +%% @private +%% @doc Convert process state when code is changed +-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, + Extra :: term()) -> + {ok, NewState :: #state{}} | {error, Reason :: term()}). +code_change(_OldVsn, State = #state{}, _Extra) -> + {ok, State}. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== + +-spec format(binary() | [iodata()]) -> binary(). +format(Data) when is_binary(Data) -> + iolist_to_binary(Data); +format(Items) when is_list(Items) -> + iolist_to_binary(lists:join(<<"\t">>, Items)). + +-spec time_prefix() -> binary(). +time_prefix() -> + {{Y, M, D}, {H, I, S}} = calendar:local_time(), + iolist_to_binary(io_lib:format("[~b-~2..0b-~2..0b ~2..0b:~2..0b:~2..0b]", [Y, M, D, H, I, S])). + +-spec make_file(LogFile :: string()) -> string(). +make_file(LogFile) when is_list(LogFile) -> + {Year, Month, Day} = erlang:date(), + Suffix = io_lib:format("~b~2..0b~2..0b", [Year, Month, Day]), + RootDir = code:root_dir() ++ "/log/", + lists:flatten(RootDir ++ LogFile ++ "." ++ Suffix). + +-spec ensure_dir() -> ok | {error, term()}. +ensure_dir() -> + RootDir = code:root_dir() ++ "/log/", + case filelib:is_dir(RootDir) of + true -> + ok; + false -> + file:make_dir(RootDir) + end. + +%% 获取日期信息 +-spec get_date() -> Date :: calendar:date(). +get_date() -> + {Date, _} = calendar:local_time(), + Date. + +%% 通过日志判断是否需要生成新的日志文件 +-spec maybe_new_file(Date :: calendar:date()) -> boolean(). +maybe_new_file({Y, M, D}) -> + {{Y0, M0, D0}, _} = calendar:local_time(), + not (Y =:= Y0 andalso M =:= M0 andalso D =:= D0). diff --git a/apps/efka/src/efka_manifest.erl b/apps/efka/src/efka_manifest.erl deleted file mode 100644 index f4db6ce..0000000 --- a/apps/efka/src/efka_manifest.erl +++ /dev/null @@ -1,124 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% 用于管理manifest.json配置文件 -%%% @end -%%% Created : 05. 5月 2025 22:39 -%%%------------------------------------------------------------------- --module(efka_manifest). --author("anlicheng"). - --record(manifest, { - work_dir = "" :: string(), - id = <<"">> :: binary(), - exec = <<"">>:: binary(), - args = [], - health_check = <<"">> -}). - --type manifest() :: #manifest{}. - --export_type([manifest/0]). - -%% API --export([new/1, startup/1]). - --spec new(ServiceRootDir :: string()) -> {ok, #manifest{}} | {error, Reason :: binary()}. -new(ServiceRootDir) when is_list(ServiceRootDir) -> - WorkDir = ServiceRootDir ++ "/work_dir/", - case file:read_file(WorkDir ++ "manifest.json") of - {ok, ManifestInfo} -> - Settings = catch jiffy:decode(ManifestInfo, [return_maps]), - case check_manifest(Settings) of - {ok, Manifest} -> - {ok, Manifest#manifest{work_dir = WorkDir}}; - {error, Reason} -> - {error, Reason} - end; - {error, Reason} -> - {error, Reason} - end. - --spec startup(Manifest :: #manifest{}) -> {ok, Port :: port()} | {error, Reason :: binary()}. -startup(#manifest{id = Id, work_dir = WorkDir, exec = ExecCmd0, args = Args0}) -> - PortSettings = [ - {cd, WorkDir}, - {args, [binary_to_list(A) || A <- Args0]}, - exit_status - ], - ExecCmd = binary_to_list(ExecCmd0), - RealExecCmd = filename:absname_join(WorkDir, ExecCmd), - lager:debug("[efka_manifest] service_id: ~p, real command is: ~p", [Id, RealExecCmd]), - case catch erlang:open_port({spawn_executable, RealExecCmd}, PortSettings) of - Port when is_port(Port) -> - {ok, Port}; - _Other -> - {error, <<"exec command startup failed">>} - end. - -%% 检查配置是否合法 --spec check_manifest(Manifest :: map()) -> {ok, #manifest{}} | {error, Reason :: binary()}. -check_manifest(Manifest) when is_map(Manifest) -> - RequiredKeys = [<<"id">>, <<"exec">>, <<"args">>, <<"health_check">>], - check_manifest0(RequiredKeys, Manifest, #manifest{}); -check_manifest(_Manifest) -> - {error, <<"invalid manifest json">>}. - -check_manifest0([], _Settings, Manifest) -> - {ok, Manifest}; -check_manifest0([<<"id">>|T], Settings, Manifest) -> - case maps:find(<<"id">>, Settings) of - error -> - {error, <<"miss service_id">>}; - {ok, Id} when is_binary(Id) -> - check_manifest0(T, Settings, Manifest#manifest{id = Id}); - {ok, _} -> - {error, <<"service_id is not string">>} - end; -check_manifest0([<<"health_check">>|T], Settings, Manifest) -> - case maps:find(<<"health_check">>, Settings) of - error -> - {error, <<"miss health_check">>}; - {ok, Url} when is_binary(Url) -> - case is_url(Url) of - true -> - check_manifest0(T, Settings, Manifest#manifest{health_check = Url}); - false -> - {error, <<"health_check is not a invalid url">>} - end; - {ok, _} -> - {error, <<"health_check is not string">>} - end; -check_manifest0([<<"exec">>|T], Settings, Manifest) -> - case maps:find(<<"exec">>, Settings) of - error -> - {error, <<"miss start">>}; - {ok, Exec} when is_binary(Exec) -> - %% 不能包含空格 - case binary:match(Exec, <<" ">>) of - nomatch -> - check_manifest0(T, Settings, Manifest#manifest{exec = Exec}); - _ -> - {error, <<"start cmd cannot contain args">>} - end - end; -check_manifest0([<<"args">>|T], Settings, Manifest) -> - case maps:find(<<"args">>, Settings) of - error -> - check_manifest0(T, Settings, Manifest#manifest{args = []}); - %% 对参数项目不进行检查 - {ok, Args} when is_list(Args) -> - check_manifest0(T, Settings, Manifest#manifest{args = Args}); - {ok, _} -> - {error, <<"args must be list">>} - end. - --spec is_url(binary()) -> boolean(). -is_url(Input) when is_binary(Input) -> - try - uri_string:parse(Input), - true - catch - _:_ -> false - end. \ No newline at end of file diff --git a/apps/efka/src/efka_monitor.erl b/apps/efka/src/efka_monitor.erl deleted file mode 100644 index 7ed3931..0000000 --- a/apps/efka/src/efka_monitor.erl +++ /dev/null @@ -1,25 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 03. 6月 2025 14:09 -%%%------------------------------------------------------------------- --module(efka_monitor). --author("anlicheng"). - -%% API --export([]). - -%% API --export([memory_top/1, cpu_top/1, stop/0]). - -memory_top(Interval) when is_integer(Interval) -> - spawn(fun()->etop:start([{output, text}, {interval, Interval}, {lines, 20}, {sort, memory}])end). - -cpu_top(Interval) when is_integer(Interval) -> - spawn(fun()->etop:start([{output, text}, {interval, Interval}, {lines, 20}, {sort, runtime}])end). - -stop() -> - etop:stop(). \ No newline at end of file diff --git a/apps/efka/src/efka_service.erl b/apps/efka/src/efka_service.erl deleted file mode 100644 index b316d7f..0000000 --- a/apps/efka/src/efka_service.erl +++ /dev/null @@ -1,291 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% 1. 需要管理服务的整个生命周期,包括: 启动,停止 -%%% 2. 需要监控服务的状态,通过port的方式 -%%% 3. 服务的启动和关闭,需要在更高的层级控制 -%%% @end -%%% Created : 18. 4月 2025 16:50 -%%%------------------------------------------------------------------- --module(efka_service). --author("anlicheng"). --include("efka_tables.hrl"). - --behaviour(gen_server). - -%% API --export([start_link/2]). --export([get_name/1, get_pid/1, attach_channel/2]). --export([push_config/3, request_config/1, invoke/3]). --export([metric_data/3, send_event/3]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). - --record(state, { - service_id :: binary(), - %% 通道id信息 - channel_pid :: pid() | undefined, - %% 当前进程的port信息, OSPid = erlang:port_info(Port, os_pid) - port :: undefined | port(), - %% 系统对应的pid - os_pid :: undefined | integer(), - %% 配置信息 - manifest :: undefined | efka_manifest:manifest(), - inflight = #{}, - - %% 映射关系: #{Ref => Fun} - callbacks = #{} -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - --spec get_name(ServiceId :: binary()) -> atom(). -get_name(ServiceId) when is_binary(ServiceId) -> - list_to_atom("efka_service:" ++ binary_to_list(ServiceId)). - --spec get_pid(ServiceId :: binary()) -> undefined | pid(). -get_pid(ServiceId) when is_binary(ServiceId) -> - whereis(get_name(ServiceId)). - --spec push_config(Pid :: pid(), Ref :: reference(), ConfigJson :: binary()) -> no_return(). -push_config(Pid, Ref, ConfigJson) when is_pid(Pid), is_binary(ConfigJson) -> - gen_server:cast(Pid, {push_config, Ref, self(), ConfigJson}). - --spec invoke(Pid :: pid(), Ref :: reference(), Payload :: binary()) -> no_return(). -invoke(Pid, Ref, Payload) when is_pid(Pid), is_reference(Ref), is_binary(Payload) -> - gen_server:cast(Pid, {invoke, Ref, self(), Payload}). - --spec request_config(Pid :: pid()) -> {ok, Config :: binary()}. -request_config(Pid) when is_pid(Pid) -> - gen_server:call(Pid, request_config). - --spec metric_data(Pid :: pid(), DeviceUUID :: binary(), Data :: binary()) -> no_return(). -metric_data(Pid, DeviceUUID, Data) when is_pid(Pid), is_binary(DeviceUUID), is_binary(Data) -> - gen_server:cast(Pid, {metric_data, DeviceUUID, Data}). - --spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> no_return(). -send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) -> - gen_server:cast(Pid, {send_event, EventType, Params}). - --spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}. -attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) -> - gen_server:call(Pid, {attach_channel, ChannelPid}). - -%% @doc Spawns the server and registers the local name (unique) --spec(start_link(Name :: atom(), Service :: binary()) -> - {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). -start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) -> - gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []). - -%%%=================================================================== -%%% gen_server callbacks -%%%=================================================================== - -%% @private -%% @doc Initializes the server --spec(init(Args :: term()) -> - {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | - {stop, Reason :: term()} | ignore). -init([ServiceId]) -> - %% supervisor进程通过exit(ChildPid, shutdown)调用的时候,确保terminate函数被调用 - erlang:process_flag(trap_exit, true), - case service_model:get_service(ServiceId) of - error -> - lager:notice("[efka_service] service_id: ~p, not found", [ServiceId]), - ignore; - {ok, #service{root_dir = RootDir}} -> - %% 第一次启动,要求必须成功;只有第一次启动成功,后续的重启逻辑才有意义 - case efka_manifest:new(RootDir) of - {ok, Manifest} -> - case efka_manifest:startup(Manifest) of - {ok, Port} -> - {os_pid, OSPid} = erlang:port_info(Port, os_pid), - lager:debug("[efka_service] service: ~p, port: ~p, boot_service success os_pid: ~p", [ServiceId, Port, OSPid]), - {ok, #state{service_id = ServiceId, manifest = Manifest, port = Port, os_pid = OSPid}}; - {error, Reason} -> - lager:debug("[efka_service] service: ~p, boot_service get error: ~p", [ServiceId, Reason]), - {stop, Reason} - end; - {error, Reason} -> - lager:notice("[efka_service] service: ~p, read manifest.json get error: ~p", [ServiceId, Reason]), - ignore - end - end. - -%% @private -%% @doc Handling call messages --spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, - State :: #state{}) -> - {reply, Reply :: term(), NewState :: #state{}} | - {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 绑定channel -handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) -> - case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of - false -> - erlang:monitor(process, ChannelPid), - lager:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]), - {reply, ok, State#state{channel_pid = ChannelPid}}; - true -> - {reply, {error, <<"channel exists">>}, State} - end; - -%% 请求参数项 done -handle_call(request_config, _From, State = #state{service_id = ServiceId}) -> - case service_model:get_config_json(ServiceId) of - {ok, ConfigJson} -> - {reply, {ok, ConfigJson}, State}; - error -> - {reply, {ok, <<>>}, State} - end; - -handle_call(_Request, _From, State = #state{}) -> - {reply, ok, State}. - -%% @private -%% @doc Handling cast messages --spec(handle_cast(Request :: term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_cast({metric_data, DeviceUUID, LineProtocolData}, State = #state{service_id = ServiceId}) -> - lager:debug("[efka_service] metric_data service_id: ~p, device_uuid: ~p, metric data: ~p", [ServiceId, DeviceUUID, LineProtocolData]), - efka_agent:metric_data(ServiceId, DeviceUUID, LineProtocolData), - {noreply, State}; - -handle_cast({send_event, EventType, Params}, State = #state{service_id = ServiceId}) -> - efka_agent:event(ServiceId, EventType, Params), - lager:debug("[efka_service] send_event, service_id: ~p, event_type: ~p, params: ~p", [ServiceId, EventType, Params]), - {noreply, State}; - -%% 推送配置项目 -handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{channel_pid = ChannelPid, service_id = ServiceId, inflight = Inflight, callbacks = Callbacks}) -> - case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of - true -> - efka_tcp_channel:push_config(ChannelPid, Ref, self(), ConfigJson), - %% 设置成功,需要更新微服务的配置 - CB = fun() -> service_model:set_config(ServiceId, ConfigJson) end, - {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight), callbacks = maps:put(Ref, CB, Callbacks)}}; - false -> - ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}}, - {noreply, State} - end; - -%% 推送配置项目 -handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{channel_pid = ChannelPid, inflight = Inflight}) -> - case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of - true -> - efka_tcp_channel:invoke(ChannelPid, Ref, self(), Payload), - {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight)}}; - false -> - ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}}, - {reply, State} - end; - -handle_cast(_Request, State = #state{}) -> - {noreply, State}. - -%% @private -%% @doc Handling all non call/cast messages --spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 重启服务 -handle_info({timeout, _, reboot_service}, State = #state{service_id = ServiceId, manifest = Manifest}) -> - case efka_manifest:startup(Manifest) of - {ok, Port} -> - {os_pid, OSPid} = erlang:port_info(Port, os_pid), - lager:debug("[efka_service] service_id: ~p, reboot success, port: ~p, os_pid: ~p", [ServiceId, Port, OSPid]), - {noreply, State#state{port = Port, os_pid = OSPid}}; - {error, Reason} -> - lager:debug("[efka_service] service_id: ~p, boot_service get error: ~p", [ServiceId, Reason]), - try_reboot(), - {noreply, State} - end; - -%% 处理channel的回复 -handle_info({channel_reply, Ref, Reply}, State = #state{inflight = Inflight, callbacks = Callbacks}) -> - case maps:take(Ref, Inflight) of - error -> - {noreply, State}; - {ReceiverPid, NInflight} -> - ReceiverPid ! {service_reply, Ref, Reply}, - - {noreply, State#state{inflight = NInflight, callbacks = trigger_callback(Ref, Callbacks)}} - end; - -handle_info({Port, {data, Data}}, State = #state{service_id = ServiceId}) when is_port(Port) -> - lager:debug("[efka_service] service_id: ~p, port data: ~p", [ServiceId, Data]), - {noreply, State}; - -%% 处理port的消息, Port的被动关闭会触发;因此这个时候的Port和State.port的值是相等的 -handle_info({Port, {exit_status, Code}}, State = #state{service_id = ServiceId}) when is_port(Port) -> - lager:debug("[efka_service] service_id: ~p, port: ~p, exit with code: ~p", [ServiceId, Port, Code]), - {noreply, State#state{port = undefined, os_pid = undefined}}; - -%% 处理port的退出消息 -handle_info({'EXIT', Port, Reason}, State = #state{service_id = ServiceId}) when is_port(Port) -> - lager:debug("[efka_service] service_id: ~p, port: ~p, exit with reason: ~p", [ServiceId, Port, Reason]), - try_reboot(), - {noreply, State#state{port = undefined, os_pid = undefined}}; - -%% 处理channel进程的退出 -handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) -> - lager:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]), - {noreply, State#state{channel_pid = undefined, inflight = #{}}}. - -%% @private -%% @doc This function is called by a gen_server when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_server terminates -%% with Reason. The return value is ignored. --spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), - State :: #state{}) -> term()). -terminate(Reason, _State = #state{service_id = ServiceId, port = Port, os_pid = OSPid}) -> - erlang:is_port(Port) andalso erlang:port_close(Port), - catch kill_os_pid(OSPid), - lager:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]), - ok. - -%% @private -%% @doc Convert process state when code is changed --spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, - Extra :: term()) -> - {ok, NewState :: #state{}} | {error, Reason :: term()}). -code_change(_OldVsn, State = #state{}, _Extra) -> - {ok, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== - -%% 关闭系统进程 --spec kill_os_pid(port() | undefined) -> no_return(). -kill_os_pid(undefined) -> - ok; -kill_os_pid(OSPid) when is_integer(OSPid) -> - Cmd = lists:flatten(io_lib:format("kill -9 ~p", [OSPid])), - lager:debug("kill cmd is: ~p", [Cmd]), - os:cmd(Cmd). - --spec try_reboot() -> no_return(). -try_reboot() -> - erlang:start_timer(5000, self(), reboot_service). - --spec trigger_callback(Ref :: reference(), Callbacks :: map()) -> NewCallbacks :: map(). -trigger_callback(Ref, Callbacks) -> - case maps:take(Ref, Callbacks) of - error -> - Callbacks; - {Fun, NCallbacks} -> - catch Fun(), - NCallbacks - end. \ No newline at end of file diff --git a/apps/efka/src/efka_std_modbus_service.erl b/apps/efka/src/efka_std_modbus_service.erl deleted file mode 100644 index 15fdb23..0000000 --- a/apps/efka/src/efka_std_modbus_service.erl +++ /dev/null @@ -1,291 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% 1. 需要管理服务的整个生命周期,包括: 启动,停止 -%%% 2. 需要监控服务的状态,通过port的方式 -%%% 3. 服务的启动和关闭,需要在更高的层级控制 -%%% @end -%%% Created : 18. 4月 2025 16:50 -%%%------------------------------------------------------------------- --module(efka_std_modbus_service). --author("anlicheng"). --include("efka_tables.hrl"). - --behaviour(gen_server). - -%% API --export([start_link/2]). --export([get_name/1, get_pid/1, attach_channel/2]). --export([push_config/3, request_config/1, invoke/3]). --export([metric_data/3, send_event/3]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). - --record(state, { - service_id :: binary(), - %% 通道id信息 - channel_pid :: pid() | undefined, - %% 当前进程的port信息, OSPid = erlang:port_info(Port, os_pid) - port :: undefined | port(), - %% 系统对应的pid - os_pid :: undefined | integer(), - %% 配置信息 - manifest :: undefined | efka_manifest:manifest(), - inflight = #{}, - - %% 映射关系: #{Ref => Fun} - callbacks = #{} -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - --spec get_name(ServiceId :: binary()) -> atom(). -get_name(ServiceId) when is_binary(ServiceId) -> - list_to_atom("efka_service:" ++ binary_to_list(ServiceId)). - --spec get_pid(ServiceId :: binary()) -> undefined | pid(). -get_pid(ServiceId) when is_binary(ServiceId) -> - whereis(get_name(ServiceId)). - --spec push_config(Pid :: pid(), Ref :: reference(), ConfigJson :: binary()) -> no_return(). -push_config(Pid, Ref, ConfigJson) when is_pid(Pid), is_binary(ConfigJson) -> - gen_server:cast(Pid, {push_config, Ref, self(), ConfigJson}). - --spec invoke(Pid :: pid(), Ref :: reference(), Payload :: binary()) -> no_return(). -invoke(Pid, Ref, Payload) when is_pid(Pid), is_reference(Ref), is_binary(Payload) -> - gen_server:cast(Pid, {invoke, Ref, self(), Payload}). - --spec request_config(Pid :: pid()) -> {ok, Config :: binary()}. -request_config(Pid) when is_pid(Pid) -> - gen_server:call(Pid, request_config). - --spec metric_data(Pid :: pid(), DeviceUUID :: binary(), Data :: binary()) -> no_return(). -metric_data(Pid, DeviceUUID, Data) when is_pid(Pid), is_binary(DeviceUUID), is_binary(Data) -> - gen_server:cast(Pid, {metric_data, DeviceUUID, Data}). - --spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> no_return(). -send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) -> - gen_server:cast(Pid, {send_event, EventType, Params}). - --spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}. -attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) -> - gen_server:call(Pid, {attach_channel, ChannelPid}). - -%% @doc Spawns the server and registers the local name (unique) --spec(start_link(Name :: atom(), Service :: binary()) -> - {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). -start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) -> - gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []). - -%%%=================================================================== -%%% gen_server callbacks -%%%=================================================================== - -%% @private -%% @doc Initializes the server --spec(init(Args :: term()) -> - {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | - {stop, Reason :: term()} | ignore). -init([ServiceId]) -> - %% supervisor进程通过exit(ChildPid, shutdown)调用的时候,确保terminate函数被调用 - erlang:process_flag(trap_exit, true), - case service_model:get_service(ServiceId) of - error -> - lager:notice("[efka_service] service_id: ~p, not found", [ServiceId]), - ignore; - {ok, #service{root_dir = RootDir}} -> - %% 第一次启动,要求必须成功;只有第一次启动成功,后续的重启逻辑才有意义 - case efka_manifest:new(RootDir) of - {ok, Manifest} -> - case efka_manifest:startup(Manifest) of - {ok, Port} -> - {os_pid, OSPid} = erlang:port_info(Port, os_pid), - lager:debug("[efka_service] service: ~p, port: ~p, boot_service success os_pid: ~p", [ServiceId, Port, OSPid]), - {ok, #state{service_id = ServiceId, manifest = Manifest, port = Port, os_pid = OSPid}}; - {error, Reason} -> - lager:debug("[efka_service] service: ~p, boot_service get error: ~p", [ServiceId, Reason]), - {stop, Reason} - end; - {error, Reason} -> - lager:notice("[efka_service] service: ~p, read manifest.json get error: ~p", [ServiceId, Reason]), - ignore - end - end. - -%% @private -%% @doc Handling call messages --spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, - State :: #state{}) -> - {reply, Reply :: term(), NewState :: #state{}} | - {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 绑定channel -handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) -> - case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of - false -> - erlang:monitor(process, ChannelPid), - lager:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]), - {reply, ok, State#state{channel_pid = ChannelPid}}; - true -> - {reply, {error, <<"channel exists">>}, State} - end; - -%% 请求参数项 done -handle_call(request_config, _From, State = #state{service_id = ServiceId}) -> - case service_model:get_config_json(ServiceId) of - {ok, ConfigJson} -> - {reply, {ok, ConfigJson}, State}; - error -> - {reply, {ok, <<>>}, State} - end; - -handle_call(_Request, _From, State = #state{}) -> - {reply, ok, State}. - -%% @private -%% @doc Handling cast messages --spec(handle_cast(Request :: term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_cast({metric_data, DeviceUUID, LineProtocolData}, State = #state{service_id = ServiceId}) -> - lager:debug("[efka_service] metric_data service_id: ~p, device_uuid: ~p, metric data: ~p", [ServiceId, DeviceUUID, LineProtocolData]), - efka_agent:metric_data(ServiceId, DeviceUUID, LineProtocolData), - {noreply, State}; - -handle_cast({send_event, EventType, Params}, State = #state{service_id = ServiceId}) -> - efka_agent:event(ServiceId, EventType, Params), - lager:debug("[efka_service] send_event, service_id: ~p, event_type: ~p, params: ~p", [ServiceId, EventType, Params]), - {noreply, State}; - -%% 推送配置项目 -handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{channel_pid = ChannelPid, service_id = ServiceId, inflight = Inflight, callbacks = Callbacks}) -> - case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of - true -> - efka_tcp_channel:push_config(ChannelPid, Ref, self(), ConfigJson), - %% 设置成功,需要更新微服务的配置 - CB = fun() -> service_model:set_config(ServiceId, ConfigJson) end, - {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight), callbacks = maps:put(Ref, CB, Callbacks)}}; - false -> - ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}}, - {noreply, State} - end; - -%% 推送配置项目 -handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{channel_pid = ChannelPid, inflight = Inflight}) -> - case is_pid(ChannelPid) andalso is_process_alive(ChannelPid) of - true -> - efka_tcp_channel:invoke(ChannelPid, Ref, self(), Payload), - {noreply, State#state{inflight = maps:put(Ref, ReceiverPid, Inflight)}}; - false -> - ReceiverPid ! {service_reply, Ref, {error, <<"channel is not alive">>}}, - {reply, State} - end; - -handle_cast(_Request, State = #state{}) -> - {noreply, State}. - -%% @private -%% @doc Handling all non call/cast messages --spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 重启服务 -handle_info({timeout, _, reboot_service}, State = #state{service_id = ServiceId, manifest = Manifest}) -> - case efka_manifest:startup(Manifest) of - {ok, Port} -> - {os_pid, OSPid} = erlang:port_info(Port, os_pid), - lager:debug("[efka_service] service_id: ~p, reboot success, port: ~p, os_pid: ~p", [ServiceId, Port, OSPid]), - {noreply, State#state{port = Port, os_pid = OSPid}}; - {error, Reason} -> - lager:debug("[efka_service] service_id: ~p, boot_service get error: ~p", [ServiceId, Reason]), - try_reboot(), - {noreply, State} - end; - -%% 处理channel的回复 -handle_info({channel_reply, Ref, Reply}, State = #state{inflight = Inflight, callbacks = Callbacks}) -> - case maps:take(Ref, Inflight) of - error -> - {noreply, State}; - {ReceiverPid, NInflight} -> - ReceiverPid ! {service_reply, Ref, Reply}, - - {noreply, State#state{inflight = NInflight, callbacks = trigger_callback(Ref, Callbacks)}} - end; - -handle_info({Port, {data, Data}}, State = #state{service_id = ServiceId}) when is_port(Port) -> - lager:debug("[efka_service] service_id: ~p, port data: ~p", [ServiceId, Data]), - {noreply, State}; - -%% 处理port的消息, Port的被动关闭会触发;因此这个时候的Port和State.port的值是相等的 -handle_info({Port, {exit_status, Code}}, State = #state{service_id = ServiceId}) when is_port(Port) -> - lager:debug("[efka_service] service_id: ~p, port: ~p, exit with code: ~p", [ServiceId, Port, Code]), - {noreply, State#state{port = undefined, os_pid = undefined}}; - -%% 处理port的退出消息 -handle_info({'EXIT', Port, Reason}, State = #state{service_id = ServiceId}) when is_port(Port) -> - lager:debug("[efka_service] service_id: ~p, port: ~p, exit with reason: ~p", [ServiceId, Port, Reason]), - try_reboot(), - {noreply, State#state{port = undefined, os_pid = undefined}}; - -%% 处理channel进程的退出 -handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) -> - lager:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]), - {noreply, State#state{channel_pid = undefined, inflight = #{}}}. - -%% @private -%% @doc This function is called by a gen_server when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_server terminates -%% with Reason. The return value is ignored. --spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), - State :: #state{}) -> term()). -terminate(Reason, _State = #state{service_id = ServiceId, port = Port, os_pid = OSPid}) -> - erlang:is_port(Port) andalso erlang:port_close(Port), - catch kill_os_pid(OSPid), - lager:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]), - ok. - -%% @private -%% @doc Convert process state when code is changed --spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, - Extra :: term()) -> - {ok, NewState :: #state{}} | {error, Reason :: term()}). -code_change(_OldVsn, State = #state{}, _Extra) -> - {ok, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== - -%% 关闭系统进程 --spec kill_os_pid(port() | undefined) -> no_return(). -kill_os_pid(undefined) -> - ok; -kill_os_pid(OSPid) when is_integer(OSPid) -> - Cmd = lists:flatten(io_lib:format("kill -9 ~p", [OSPid])), - lager:debug("kill cmd is: ~p", [Cmd]), - os:cmd(Cmd). - --spec try_reboot() -> no_return(). -try_reboot() -> - erlang:start_timer(5000, self(), reboot_service). - --spec trigger_callback(Ref :: reference(), Callbacks :: map()) -> NewCallbacks :: map(). -trigger_callback(Ref, Callbacks) -> - case maps:take(Ref, Callbacks) of - error -> - Callbacks; - {Fun, NCallbacks} -> - catch Fun(), - NCallbacks - end. \ No newline at end of file diff --git a/apps/efka/src/efka_sup.erl b/apps/efka/src/efka_sup.erl index 212e443..1bdb48e 100644 --- a/apps/efka/src/efka_sup.erl +++ b/apps/efka/src/efka_sup.erl @@ -13,6 +13,7 @@ -define(SERVER, ?MODULE). +-spec start_link() -> {ok, pid()} | ignore | {error, term()}. start_link() -> supervisor:start_link({local, ?SERVER}, ?MODULE, []). @@ -25,16 +26,35 @@ start_link() -> %% shutdown => shutdown(), % optional %% type => worker(), % optional %% modules => modules()} % optional +-spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}. init([]) -> SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600}, ChildSpecs = [ #{ - id => 'efka_inetd_task_log', - start => {'efka_inetd_task_log', start_link, []}, + id => 'efka_logger', + start => {'efka_logger', start_link, ["deploy_log"]}, restart => permanent, shutdown => 2000, type => worker, - modules => ['efka_inetd_task_log'] + modules => ['efka_logger'] + }, + + #{ + id => 'efka_service_sup', + start => {'efka_service_sup', start_link, []}, + restart => permanent, + shutdown => 2000, + type => supervisor, + modules => ['efka_service_sup'] + }, + + #{ + id => efka_service_model, + start => {efka_service_model, start_link, []}, + restart => permanent, + shutdown => 5000, + type => worker, + modules => ['efka_service_model'] }, #{ @@ -47,52 +67,25 @@ init([]) -> }, #{ - id => 'efka_inetd', - start => {'efka_inetd', start_link, []}, + id => 'efka_iot_client', + start => {'efka_iot_client', start_link, []}, restart => permanent, shutdown => 2000, type => worker, - modules => ['efka_inetd'] + modules => ['efka_iot_client'] }, #{ - id => 'efka_agent', - start => {'efka_agent', start_link, []}, + id => 'efka_iot_heartbeat', + start => {'efka_iot_heartbeat', start_link, []}, restart => permanent, shutdown => 2000, type => worker, - modules => ['efka_agent'] - }, - - #{ - id => 'efka_tcp_sup', - start => {'efka_tcp_sup', start_link, []}, - restart => permanent, - shutdown => 2000, - type => supervisor, - modules => ['efka_tcp_sup'] - }, - - #{ - id => 'efka_tcp_server', - start => {'efka_tcp_server', start_link, []}, - restart => permanent, - shutdown => 2000, - type => worker, - modules => ['efka_tcp_server'] - }, - - #{ - id => 'efka_service_sup', - start => {'efka_service_sup', start_link, []}, - restart => permanent, - shutdown => 2000, - type => supervisor, - modules => ['efka_service_sup'] + modules => ['efka_iot_heartbeat'] } + ], {ok, {SupFlags, ChildSpecs}}. %% internal functions - diff --git a/apps/efka/src/efka_tcp_channel.erl b/apps/efka/src/efka_tcp_channel.erl deleted file mode 100644 index 6a7c7f6..0000000 --- a/apps/efka/src/efka_tcp_channel.erl +++ /dev/null @@ -1,295 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 30. 4月 2025 09:22 -%%%------------------------------------------------------------------- --module(efka_tcp_channel). --author("anlicheng"). - --behaviour(gen_server). - -%% API --export([start_link/1]). --export([push_config/4, invoke/4]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). - --define(SERVER, ?MODULE). - -%% 最大的等待时间 --define(PENDING_TIMEOUT, 10 * 1000). -%% 消息类型 - -%% 服务注册 --define(PACKET_REQUEST, 16#01). -%% 消息响应 --define(PACKET_RESPONSE, 16#02). -%% 上传数据 --define(PACKET_PUSH, 16#03). - --define(PACKET_PUB, 16#04). - --record(state, { - packet_id = 1, - socket :: gen_tcp:socket(), - service_id :: undefined | binary(), - service_pid :: undefined | pid(), - is_registered = false :: boolean(), - - %% 请求响应的对应关系, #{packet_id => {ReceiverPid, Ref}}; 自身的inflight需要超时逻辑处理 - inflight = #{} -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - --spec push_config(ChannelPid :: pid(), Ref :: reference(), ReceiverPid :: pid(), ConfigJson :: binary()) -> no_return(). -push_config(ChannelPid, Ref, ReceiverPid, ConfigJson) when is_pid(ChannelPid), is_pid(ReceiverPid), is_binary(ConfigJson), is_reference(Ref) -> - gen_server:cast(ChannelPid, {push_config, Ref, ReceiverPid, ConfigJson}). - --spec invoke(ChannelPid :: pid(), Ref :: reference(), ReceiverPid :: pid(), Payload :: binary()) -> no_return(). -invoke(ChannelPid, Ref, ReceiverPid, Payload) when is_pid(ChannelPid), is_pid(ReceiverPid), is_binary(Payload), is_reference(Ref) -> - gen_server:cast(ChannelPid, {invoke, Ref, ReceiverPid, Payload}). - -%% @doc Spawns the server and registers the local name (unique) --spec(start_link(Socket :: gen_tcp:socket()) -> - {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). -start_link(Socket) -> - gen_server:start_link(?MODULE, [Socket], []). - -%%%=================================================================== -%%% gen_server callbacks -%%%=================================================================== - -%% @private -%% @doc Initializes the server --spec(init(Args :: term()) -> - {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | - {stop, Reason :: term()} | ignore). -init([Socket]) -> - ok = inet:setopts(Socket, [{active, true}]), - lager:debug("[efka_tcp_channel] get micro service socket: ~p", [Socket]), - {ok, #state{socket = Socket}}. - -%% @private -%% @doc Handling call messages --spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, - State :: #state{}) -> - {reply, Reply :: term(), NewState :: #state{}} | - {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_call(_Request, _From, State = #state{}) -> - {reply, ok, State}. - -%% @private -%% @doc Handling cast messages --spec(handle_cast(Request :: term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 推送配置项目 -handle_cast({push_config, Ref, ReceiverPid, ConfigJson}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) -> - PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"push_config">>, <<"params">> => #{<<"config">> => ConfigJson}}, - Packet = jiffy:encode(PushConfig, [force_utf8]), - ok = gen_tcp:send(Socket, <>), - - erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}), - {noreply, State#state{packet_id = next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}}; - -%% 远程调用 -handle_cast({invoke, Ref, ReceiverPid, Payload}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) -> - PushConfig = #{<<"id">> => PacketId, <<"method">> => <<"invoke">>, <<"params">> => #{<<"payload">> => Payload}}, - Packet = jiffy:encode(PushConfig, [force_utf8]), - ok = gen_tcp:send(Socket, <>), - - erlang:start_timer(?PENDING_TIMEOUT, self(), {pending_timeout, PacketId}), - {noreply, State#state{packet_id = next_packet_id(PacketId), inflight = maps:put(PacketId, {ReceiverPid, Ref}, Inflight)}}; - -handle_cast(_Request, State = #state{}) -> - {noreply, State}. - -%% @private -%% @doc Handling all non call/cast messages --spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 处理micro-client:request => efka 主动的请求 -handle_info({tcp, Socket, <>}, State = #state{socket = Socket}) -> - Request = jiffy:decode(Data, [return_maps]), - case handle_request(Request, State) of - {ok, NewState} -> - {noreply, NewState}; - {stop, Reason, NewState} -> - {stop, Reason, NewState} - end; - -%% 处理micro-client:response => efka 的响应 -handle_info({tcp, Socket, <>}, State = #state{socket = Socket, inflight = Inflight}) -> - Resp = jiffy:decode(Data, [return_maps]), - case Resp of - #{<<"id">> := Id, <<"result">> := Result} -> - case maps:take(Id, Inflight) of - error -> - lager:warning("[tcp_channel] get unknown publish response message: ~p, packet_id: ~p", [Resp, Id]), - {noreply, State}; - {{ReceiverPid, Ref}, NInflight} -> - case is_pid(ReceiverPid) andalso is_process_alive(ReceiverPid) of - true -> - ReceiverPid ! {channel_reply, Ref, {ok, Result}}; - false -> - lager:warning("[tcp_channel] get publish response message: ~p, packet_id: ~p, but receiver_pid is deaded", [Resp, Id]) - end, - {noreply, State#state{inflight = NInflight}} - end; - #{<<"id">> := Id, <<"error">> := #{<<"code">> := _Code, <<"message">> := Error}} -> - case maps:take(Id, Inflight) of - error -> - lager:warning("[tcp_channel] get unknown publish response message: ~p, packet_id: ~p", [Resp, Id]), - {noreply, State}; - {{ReceiverPid, Ref}, NInflight} -> - case is_pid(ReceiverPid) andalso is_process_alive(ReceiverPid) of - true -> - ReceiverPid ! {channel_reply, Ref, {error, Error}}; - false -> - lager:warning("[tcp_channel] get publish response message: ~p, packet_id: ~p, but receiver_pid is deaded", [Resp, Id]) - end, - {noreply, State#state{inflight = NInflight}} - end - end; - -%% 超时逻辑处理 -handle_info({timeout, _, {pending_timeout, Id}}, State = #state{inflight = Inflight}) -> - case maps:take(Id, Inflight) of - error -> - {noreply, State}; - {{ReceiverPid, Ref}, NInflight} -> - case is_pid(ReceiverPid) andalso is_process_alive(ReceiverPid) of - true -> - ReceiverPid ! {channel_reply, Ref, {error, <<"timeout">>}}; - false -> - ok - end, - {noreply, State#state{inflight = NInflight}} - end; - -%% 订阅的消息 -handle_info({topic_broadcast, Topic, Content}, State = #state{socket = Socket}) -> - Packet = jiffy:encode(#{<<"topic">> => Topic, <<"content">> => Content}, [force_utf8]), - ok = gen_tcp:send(Socket, <>), - - {noreply, State}; - -%% service进程关闭 -handle_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) -> - lager:debug("[tcp_channel] service_pid: ~p, exited: ~p", [ServicePid, Reason]), - {stop, normal, State#state{service_pid = undefined}}; - -handle_info({tcp_error, Socket, Reason}, State = #state{socket = Socket, service_id = ServiceId}) -> - lager:debug("[tcp_channel] tcp_error: ~p, assoc service: ~p", [Reason, ServiceId]), - {stop, normal, State}; -handle_info({tcp_closed, Socket}, State = #state{socket = Socket, service_id = ServiceId}) -> - lager:debug("[tcp_channel] tcp_closed: ~p, assoc service: ~p", [Socket, ServiceId]), - {stop, normal, State}. - -%% @private -%% @doc This function is called by a gen_server when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_server terminates -%% with Reason. The return value is ignored. --spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), - State :: #state{}) -> term()). -terminate(_Reason, _State = #state{}) -> - ok. - -%% @private -%% @doc Convert process state when code is changed --spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, - Extra :: term()) -> - {ok, NewState :: #state{}} | {error, Reason :: term()}). -code_change(_OldVsn, State = #state{}, _Extra) -> - {ok, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== - -%% 注册 -handle_request(#{<<"id">> := Id, <<"method">> := <<"register">>, <<"params">> := #{<<"service_id">> := ServiceId}}, State = #state{socket = Socket}) -> - case efka_service:get_pid(ServiceId) of - undefined -> - lager:warning("[efka_tcp_channel] service_id: ~p, not running", [ServiceId]), - Packet = json_error(Id, -1, <<"service not running">>), - ok = gen_tcp:send(Socket, <>), - {stop, normal, State}; - ServicePid when is_pid(ServicePid) -> - case efka_service:attach_channel(ServicePid, self()) of - ok -> - Packet = json_result(Id, <<"ok">>), - erlang:monitor(process, ServicePid), - - ok = gen_tcp:send(Socket, <>), - {ok, State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}}; - {error, Error} -> - lager:warning("[efka_tcp_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]), - Packet = json_error(Id, -1, Error), - ok = gen_tcp:send(Socket, <>), - {stop, normal, State} - end - end; - -%% 请求参数 -handle_request(#{<<"id">> := Id, <<"method">> := <<"request_config">>}, State = #state{socket = Socket, service_pid = ServicePid, is_registered = true}) -> - {ok, ConfigJson} = efka_service:request_config(ServicePid), - Packet = json_result(Id, ConfigJson), - ok = gen_tcp:send(Socket, <>), - {ok, State}; - -%% 数据项 -handle_request(#{<<"id">> := 0, <<"method">> := <<"metric_data">>, <<"params">> := #{<<"device_uuid">> := DeviceUUID, <<"metric">> := Metric}}, State = #state{service_pid = ServicePid, is_registered = true}) -> - efka_service:metric_data(ServicePid, DeviceUUID, Metric), - {ok, State}; - -%% Event事件 -handle_request(#{<<"id">> := 0, <<"method">> := <<"event">>, <<"params">> := #{<<"event_type">> := EventType, <<"body">> := Body}}, State = #state{service_pid = ServicePid, is_registered = true}) -> - efka_service:send_event(ServicePid, EventType, Body), - {ok, State}; - -%% 订阅事件 -handle_request(#{<<"id">> := 0, <<"method">> := <<"subscribe">>, <<"params">> := #{<<"topic">> := Topic}}, State = #state{is_registered = true}) -> - efka_subscription:subscribe(Topic, self()), - {ok, State}. - -%% 采用32位编码 --spec next_packet_id(PacketId :: integer()) -> NextPacketId :: integer(). -next_packet_id(PacketId) when PacketId >= 4294967295 -> - 1; -next_packet_id(PacketId) -> - PacketId + 1. - --spec json_result(Id :: integer(), Result :: term()) -> binary(). -json_result(Id, Result) when is_integer(Id) -> - Response = #{ - <<"id">> => Id, - <<"result">> => Result - }, - jiffy:encode(Response, [force_utf8]). - --spec json_error(Id :: integer(), Code :: integer(), Message :: binary()) -> binary(). -json_error(Id, Code, Message) when is_integer(Id), is_integer(Code), is_binary(Message) -> - Response = #{ - <<"id">> => Id, - <<"error">> => #{ - <<"code">> => Code, - <<"message">> => Message - } - }, - jiffy:encode(Response, [force_utf8]). \ No newline at end of file diff --git a/apps/efka/src/efka_tcp_server.erl b/apps/efka/src/efka_tcp_server.erl deleted file mode 100644 index e1646e2..0000000 --- a/apps/efka/src/efka_tcp_server.erl +++ /dev/null @@ -1,46 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 29. 4月 2025 23:24 -%%%------------------------------------------------------------------- --module(efka_tcp_server). --author("anlicheng"). - -%% API --export([start_link/0, init/0]). - -start_link() -> - {ok, spawn_link(?MODULE, init, [])}. - -%% 监听循环 -init() -> - {ok, TcpServerProps} = application:get_env(efka, tcp_server), - Port = proplists:get_value(port, TcpServerProps), - - case gen_tcp:listen(Port, [binary, {packet, 4}, {active, false}, {reuseaddr, true}]) of - {ok, ListenSocket} -> - lager:debug("[efka_tcp_server] Server started on port ~p~n", [Port]), - main_loop(ListenSocket); - {error, Reason} -> - lager:debug("[efka_tcp_server] Failed to start server: ~p~n", [Reason]), - exit(Reason) - end. - -main_loop(ListenSocket) -> - case gen_tcp:accept(ListenSocket) of - {ok, Socket} -> - % 为每个新连接生成一个处理进程 - {ok, ChannelPid} = efka_tcp_sup:start_child(Socket), - ok = gen_tcp:controlling_process(Socket, ChannelPid), - % 继续监听下一个连接 - main_loop(ListenSocket); - {error, closed} -> - lager:debug("[efka_tcp_server] Server socket closed"), - exit(tcp_closed); - {error, Reason} -> - lager:debug("[efka_tcp_server] Accept error: ~p", [Reason]), - exit(Reason) - end. \ No newline at end of file diff --git a/apps/efka/src/efka_tcp_sup.erl b/apps/efka/src/efka_tcp_sup.erl deleted file mode 100644 index 0d3325c..0000000 --- a/apps/efka/src/efka_tcp_sup.erl +++ /dev/null @@ -1,43 +0,0 @@ -%%%------------------------------------------------------------------- -%% @doc efka top level supervisor. -%% @end -%%%------------------------------------------------------------------- - --module(efka_tcp_sup). - --behaviour(supervisor). - --export([start_link/0, start_child/1]). - --export([init/1]). - --define(SERVER, ?MODULE). - -start_link() -> - supervisor:start_link({local, ?SERVER}, ?MODULE, []). - -%% sup_flags() = #{strategy => strategy(), % optional -%% intensity => non_neg_integer(), % optional -%% period => pos_integer()} % optional -%% child_spec() = #{id => child_id(), % mandatory -%% start => mfargs(), % mandatory -%% restart => restart(), % optional -%% shutdown => shutdown(), % optional -%% type => worker(), % optional -%% modules => modules()} % optional -init([]) -> - SupFlags = #{strategy => simple_one_for_one, intensity => 0, period => 1}, - ChildSpec = #{ - id => efka_tcp_channel, - start => {efka_tcp_channel, start_link, []}, - restart => temporary, - type => worker - }, - {ok, {SupFlags, [ChildSpec]}}. - -%% internal functions - -start_child(Socket) -> - supervisor:start_child(?MODULE, [Socket]). - - diff --git a/apps/efka/src/efka_transport.erl b/apps/efka/src/efka_transport.erl deleted file mode 100644 index c0641fc..0000000 --- a/apps/efka/src/efka_transport.erl +++ /dev/null @@ -1,226 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 20. 4月 2025 18:47 -%%%------------------------------------------------------------------- --module(efka_transport). --author("anlicheng"). --include("message_pb.hrl"). --include("efka.hrl"). - --behaviour(gen_server). - -%% API --export([start_monitor/3]). --export([connect/1, auth_request/2, send/3, async_call_reply/3, stop/1]). --export([request/3]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). - --define(SERVER, ?MODULE). - --record(state, { - parent_pid :: pid(), - host :: string(), - port :: integer(), - socket :: undefined | ssl:sslsocket(), - packet_id = 1, - %% 通过packet_id建立请求和响应的关系 - inflight = #{} -}). - -%%%=================================================================== -%%% API -%%%=================================================================== - --spec auth_request(Pid :: pid(), AuthBin :: binary()) -> no_return(). -auth_request(Pid, AuthBin) when is_pid(Pid), is_binary(AuthBin) -> - gen_server:cast(Pid, {auth_request, AuthBin}). - --spec request(Pid :: pid(), Method :: integer(), ReqBin :: binary()) -> Ref :: reference(). -request(Pid, Method, ReqBin) when is_pid(Pid), is_binary(ReqBin) -> - Ref = make_ref(), - gen_server:cast(Pid, {request, Ref, Method, ReqBin}), - Ref. - --spec connect(Pid :: pid()) -> no_return(). -connect(Pid) when is_pid(Pid) -> - gen_server:cast(Pid, connect). - --spec send(Pid :: pid(), Method :: integer(), Packet :: binary()) -> no_return(). -send(Pid, Method, Packet) when is_pid(Pid), is_integer(Method), is_binary(Packet) -> - gen_server:cast(Pid, {send, Method, Packet}). - --spec async_call_reply(Pid :: pid() | undefined, PacketId :: integer(), Response :: binary()) -> no_return(). -async_call_reply(undefined, PacketId, Response) when is_integer(PacketId), is_binary(Response) -> - ok; -async_call_reply(Pid, PacketId, Response) when is_pid(Pid), is_integer(PacketId), is_binary(Response) -> - gen_server:cast(Pid, {async_call_reply, PacketId, Response}). - -%% 关闭的时候不一定能成功,可能关闭的时候;transport进程已经退出了 --spec stop(Pid :: pid() | undefined) -> ok. -stop(undefined) -> - ok; -stop(Pid) when is_pid(Pid) -> - catch gen_server:stop(Pid, normal, 2000). - -%% @doc Spawns the server and registers the local name (unique) --spec(start_monitor(ParentPid :: pid(), Host :: string(), Port :: integer()) -> - {ok, {Pid :: pid(), MRef :: reference()}} | ignore | {error, Reason :: term()}). -start_monitor(ParentPid, Host, Port) when is_pid(ParentPid), is_list(Host), is_integer(Port) -> - gen_server:start_monitor(?MODULE, [ParentPid, Host, Port], []). - -%%%=================================================================== -%%% gen_server callbacks -%%%=================================================================== - -%% @private -%% @doc Initializes the server --spec(init(Args :: term()) -> - {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | - {stop, Reason :: term()} | ignore). -init([ParentPid, Host, Port]) -> - {ok, #state{parent_pid = ParentPid, host = Host, port = Port, socket = undefined}}. - -%% @private -%% @doc Handling call messages --spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, State :: #state{}) -> - {reply, Reply :: term(), NewState :: #state{}} | - {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | - {stop, Reason :: term(), NewState :: #state{}}). -handle_call(_Req, _From, State = #state{}) -> - {reply, ok, State#state{}}. - -%% @private -%% @doc Handling cast messages --spec(handle_cast(Request :: term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 建立到目标服务器的连接 -handle_cast(connect, State = #state{host = Host, port = Port, parent_pid = ParentPid}) -> - SslOptions = [ - binary, - {packet, 4}, - {verify, verify_none} - ], - case ssl:connect(Host, Port, SslOptions, 5000) of - {ok, Socket} -> - ok = ssl:controlling_process(Socket, self()), - ParentPid ! {connect_reply, ok}, - ping_ticker(), - {noreply, State#state{socket = Socket}}; - {error, Reason} -> - ParentPid ! {connect_reply, {error, Reason}}, - {noreply, State#state{socket = undefined}} - end; - -%% auth校验 -handle_cast({auth_request, AuthRequestBin}, State = #state{parent_pid = ParentPid, socket = Socket, packet_id = PacketId}) -> - ok = ssl:send(Socket, <>), - %% 需要等待auth返回的结果 - receive - {ssl, Socket, <>} -> - ParentPid ! {auth_reply, {ok, ReplyBin}}, - {noreply, State#state{packet_id = PacketId + 1}}; - {ssl, Socket, Info} -> - lager:warning("[efka_transport] get invalid auth_reply: ~p", [Info]), - ParentPid ! {auth_reply, {error, invalid_auth_reply}}, - {noreply, State#state{packet_id = PacketId + 1}} - after 5000 -> - ParentPid ! {auth_reply, {error, timeout}}, - {noreply, State#state{packet_id = PacketId + 1}} - end; - -%% 提交请求 -handle_cast({request, Ref, Method, ReqBin}, State = #state{socket = Socket, packet_id = PacketId, inflight = Inflight}) -> - ok = ssl:send(Socket, <>), - {noreply, State#state{packet_id = PacketId + 1, inflight = maps:put(PacketId, Ref, Inflight)}}; - -handle_cast({send, Method, Packet}, State = #state{socket = Socket}) -> - ok = ssl:send(Socket, <>), - {noreply, State}; - -%% 服务push的消息的回复 -handle_cast({async_call_reply, PacketId, Response}, State = #state{socket = Socket}) -> - ok = ssl:send(Socket, <>), - {noreply, State}. - -%% @private -%% @doc Handling all non call/cast messages --spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> - {noreply, NewState :: #state{}} | - {noreply, NewState :: #state{}, timeout() | hibernate} | - {stop, Reason :: term(), NewState :: #state{}}). -%% 服务器主动推送的数据,有packetId的是要求返回的;为0的表示不需要返回值 -handle_info({ssl, Socket, <>}, State = #state{socket = Socket, parent_pid = ParentPid}) -> - ParentPid ! {server_command, CommandType, Command}, - {noreply, State}; - -handle_info({ssl, Socket, <>}, State = #state{socket = Socket, parent_pid = ParentPid}) -> - #pub{topic = Topic, content = Content} = message_pb:decode_msg(PubBin, pub), - ParentPid ! {server_pub, Topic, Content}, - {noreply, State}; - -handle_info({ssl, Socket, <>}, State = #state{socket = Socket, parent_pid = ParentPid}) -> - ParentPid ! {server_async_call, PacketId, AsyncCallBin}, - {noreply, State}; - -%% efka:request <-> iot:response -handle_info({ssl, Socket, <>}, State = #state{socket = Socket, inflight = Inflight, parent_pid = ParentPid}) -> - case maps:take(PacketId, Inflight) of - error -> - {noreply, State}; - {Ref, NInflight} -> - ParentPid ! {server_reply, Ref, ReplyBin}, - {noreply, State#state{inflight = NInflight}} - end; - -handle_info({ssl_error, Socket, Reason}, State = #state{socket = Socket}) -> - lager:debug("[efka_transport] ssl error: ~p", [Reason]), - {stop, normal, State}; - -handle_info({ssl_closed, Socket}, State = #state{socket = Socket}) -> - {stop, normal, State}; - -handle_info({timeout, _, ping_ticker}, State = #state{socket = Socket}) -> - ok = ssl:send(Socket, <>), - ping_ticker(), - {noreply, State}; - -handle_info(Info, State = #state{}) -> - lager:notice("[efka_transport] get unknown info: ~p", [Info]), - {noreply, State}. - -%% @private -%% @doc This function is called by a gen_server when it is about to -%% terminate. It should be the opposite of Module:init/1 and do any -%% necessary cleaning up. When it returns, the gen_server terminates -%% with Reason. The return value is ignored. --spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), - State :: #state{}) -> term()). -terminate(Reason, #state{}) -> - lager:notice("[efka_transport] terminate with reason: ~p", [Reason]), - ok. - -%% @private -%% @doc Convert process state when code is changed --spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, - Extra :: term()) -> - {ok, NewState :: #state{}} | {error, Reason :: term()}). -code_change(_OldVsn, State = #state{}, _Extra) -> - {ok, State}. - -%%%=================================================================== -%%% Internal functions -%%%=================================================================== - -ping_ticker() -> - erlang:start_timer(5000, self(), ping_ticker). \ No newline at end of file diff --git a/apps/efka/src/efka_util.erl b/apps/efka/src/efka_util.erl index fbe0e54..f1f4fb4 100644 --- a/apps/efka/src/efka_util.erl +++ b/apps/efka/src/efka_util.erl @@ -14,26 +14,32 @@ -export([timestamp/0, number_format/2, timestamp_ms/0, float_to_binary/2, int_format/2]). -export([chunks/2, rand_bytes/1, uuid/0, md5/1, sha_uuid/0]). -export([json_data/1, json_error/2]). +-export([starts_with/2, file_md5/1]). +-spec get_file_md5(string()) -> string(). get_file_md5(FilePath) when is_list(FilePath) -> {ok, FileData} = file:read_file(FilePath), Md5Binary = crypto:hash(md5, FileData), string:lowercase(lists:flatten([io_lib:format("~2.16.0b", [X]) || X <- binary_to_list(Md5Binary)])). %% 时间,精确到毫秒 +-spec timestamp_ms() -> integer(). timestamp_ms() -> {Mega, Seconds, Micro} = os:timestamp(), (Mega * 1000000 + Seconds) * 1000 + Micro div 1000. +-spec timestamp() -> integer(). timestamp() -> {Mega, Seconds, _Micro} = os:timestamp(), Mega * 1000000 + Seconds. +-spec number_format(integer() | float(), integer()) -> integer() | float(). number_format(Num, _Decimals) when is_integer(Num) -> Num; number_format(Float, Decimals) when is_float(Float) -> list_to_float(float_to_list(Float, [{decimals, Decimals}, compact])). +-spec int_format(integer(), pos_integer()) -> integer(). int_format(Num, Len) when is_integer(Num), Len > 0 -> S = integer_to_list(Num), case length(S) > Len of @@ -49,6 +55,8 @@ chunks(List, Size) when is_list(List), is_integer(Size), Size > 0, length(List) [List]; chunks(List, Size) when is_list(List), is_integer(Size), Size > 0 -> chunks0(List, Size, Size, [], []). + +-spec chunks0(list(), integer(), integer(), list(), [list()]) -> [list()]. chunks0([], _, _, [], AccTarget) -> lists:reverse(AccTarget); chunks0([], _, _, Target, AccTarget) -> @@ -58,19 +66,22 @@ chunks0(List, Size, 0, Target, AccTarget) -> chunks0([Hd | Tail], Size, Num, Target, AccTarget) -> chunks0(Tail, Size, Num - 1, [Hd | Target], AccTarget). +-spec json_data(term()) -> binary(). json_data(Data) -> - jiffy:encode(#{ + iolist_to_binary(json:encode(#{ <<"result">> => Data - }, [force_utf8]). + })). +-spec json_error(integer(), binary()) -> binary(). json_error(ErrCode, ErrMessage) when is_integer(ErrCode), is_binary(ErrMessage) -> - jiffy:encode(#{ + iolist_to_binary(json:encode(#{ <<"error">> => #{ <<"code">> => ErrCode, <<"message">> => ErrMessage } - }, [force_utf8]). + })). +-spec uuid() -> string(). uuid() -> rand_bytes(16). @@ -85,6 +96,7 @@ rand_bytes(Size) when is_integer(Size), Size > 0 -> md5(Str) when is_binary(Str) -> list_to_binary(lists:flatten([hex(X) || <> <= erlang:md5(Str)])). +-spec hex(0..15) -> byte(). hex(N) when N < 10 -> $0 + N; hex(N) -> @@ -102,4 +114,29 @@ float_to_binary(V, Decimals) when is_float(V), is_integer(Decimals) -> sha_uuid() -> Salt = crypto:strong_rand_bytes(32), Str = string:lowercase(binary:encode_hex(crypto:hash(sha256, Salt))), - binary:part(Str, 1, 32). \ No newline at end of file + binary:part(Str, 1, 32). + +-spec starts_with(Binary :: binary(), Prefix :: binary()) -> boolean(). +starts_with(Binary, Prefix) when is_binary(Binary), is_binary(Prefix) -> + PrefixSize = byte_size(Prefix), + case Binary of + <> -> true; + _ -> false + end. + +-spec file_md5(FilePath :: string()) -> Md5 :: string(). +file_md5(FilePath) when is_list(FilePath) -> + {ok, F} = file:open(FilePath, [read, binary]), + Digest = md5_loop(F, crypto:hash_init(md5)), + file:close(F), + lists:flatten(io_lib:format("~32.16.0b", [binary:decode_unsigned(Digest)])). + +-spec md5_loop(file:io_device(), crypto:hash_state()) -> binary(). +md5_loop(F, Context) -> + %% 每次读取 1MB,可调整块大小 + case file:read(F, 1024 * 1024) of + eof -> + crypto:hash_final(Context); + {ok, Bin} -> + md5_loop(F, crypto:hash_update(Context, Bin)) + end. diff --git a/apps/efka/src/iot/efka_iot_client.erl b/apps/efka/src/iot/efka_iot_client.erl new file mode 100644 index 0000000..eb727f6 --- /dev/null +++ b/apps/efka/src/iot/efka_iot_client.erl @@ -0,0 +1,441 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2025, +%%% @doc +%%% +%%% @end +%%% Created : 20. 4月 2026 00:00 +%%%------------------------------------------------------------------- +-module(efka_iot_client). +-author("anlicheng"). +-include("efka_tables.hrl"). + +-behaviour(gen_statem). + +%% API +-export([start_link/0]). +-export([metric_data/2, ping/13, task_event_stream/3, close_task_event_stream/2]). +-export([is_activated/0, dropped_message_count/0]). + +%% gen_statem callbacks +-export([init/1, handle_event/4, terminate/3, code_change/4, callback_mode/0]). + +-define(SERVER, ?MODULE). + +%% 标记当前agent的状态,只有在 activated 状态下才可以正常的发送数据 +-define(STATE_DISCONNECTED, disconnected). +%% 等待校验中 +-define(STATE_AUTH, auth). +%% 激活状态下 +-define(STATE_ACTIVATED, activated). +-define(SSL_PING_INTERVAL, 30000). +-define(OUTBOX_SEGMENT_RECORD_LIMIT, 2000). +-define(OUTBOX_MAX_SEGMENTS, 5). +-define(OUTBOX_MAX_RECORD_BYTES, 16 * 1024 * 1024). + +-record(state, { + socket :: undefined | ssl:sslsocket(), + outbox :: efka_iot_outbox:outbox(), + %% 保存当前auth请求的ref,用来建立auth请求和响应的对应关系 + auth_ref = undefined :: undefined | binary(), + ping_timer_ref = undefined :: undefined | reference(), + dropped_message_count = 0 :: non_neg_integer() +}). + +%%%=================================================================== +%%% API +%%%=================================================================== + +%% 发送数据 +-spec metric_data(RouteKey :: binary(), Metric :: binary()) -> ok. +metric_data(RouteKey, Metric) when is_binary(RouteKey), is_binary(Metric) -> + gen_statem:cast(?SERVER, {metric_data, RouteKey, Metric}). + +-spec task_event_stream(TaskId :: integer(), Type :: binary(), Stream :: binary()) -> ok. +task_event_stream(TaskId, Type, Stream) when is_integer(TaskId), is_binary(Type), is_binary(Stream) -> + gen_statem:cast(?SERVER, {task_event_stream, TaskId, Type, Stream}). + +-spec close_task_event_stream(TaskId :: integer(), Reason :: binary()) -> ok. +close_task_event_stream(TaskId, Reason) when is_integer(TaskId), is_binary(Reason) -> + gen_statem:cast(?SERVER, {close_task_event_stream, TaskId, Reason}). + +-spec is_activated() -> boolean(). +is_activated() -> + gen_statem:call(?SERVER, is_activated). + +-spec dropped_message_count() -> non_neg_integer(). +dropped_message_count() -> + gen_statem:call(?SERVER, dropped_message_count). + +-spec ping(term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term()) -> ok. +ping(AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces) -> + gen_statem:cast(?SERVER, {ping, AdCode, BootTime, Province, City, EfkaVersion, KernelArch, Ips, CpuCore, CpuLoad, CpuTemperature, Disk, Memory, Interfaces}). + +-spec start_link() -> {ok, pid()} | ignore | {error, term()}. +start_link() -> + gen_statem:start_link({local, ?SERVER}, ?MODULE, [], []). + +%%%=================================================================== +%%% gen_statem callbacks +%%%=================================================================== + +-spec init(list()) -> {ok, atom(), #state{}}. +init([]) -> + case efka_iot_outbox:open(outbox_options()) of + {ok, Outbox} -> + erlang:start_timer(0, self(), create_transport), + {ok, ?STATE_DISCONNECTED, #state{socket = undefined, outbox = Outbox}}; + {error, Reason} -> + {stop, Reason} + end. + +-spec callback_mode() -> handle_event_function. +callback_mode() -> + handle_event_function. + +-spec outbox_options() -> map(). +outbox_options() -> + {ok, DetsDir} = application:get_env(efka, dets_dir), + #{ + dir => filename:join(DetsDir, "iot_outbox"), + segment_record_limit => ?OUTBOX_SEGMENT_RECORD_LIMIT, + max_segments => ?OUTBOX_MAX_SEGMENTS, + max_record_bytes => ?OUTBOX_MAX_RECORD_BYTES + }. + +%% 异步发送数据,连接存在时直接发送;否则写入持久化 outbox。 +-spec handle_event(term(), term(), atom(), #state{}) -> term(). +handle_event(cast, {metric_data, RouteKey, Metric}, StateName, State = #state{socket = Socket}) -> + Packet = term_to_binary({<<"message">>, {<<"data">>, #{<<"route_key">> => RouteKey, <<"metric">> => Metric}}}), + case StateName of + ?STATE_ACTIVATED -> + ok = ssl:send(Socket, Packet), + {keep_state, State}; + _ -> + case efka_iot_outbox:append(Packet, State#state.outbox) of + {ok, Outbox} -> + {keep_state, State#state{outbox = Outbox}}; + {dropped, capacity_reached, Outbox} -> + logger:warning("[efka_iot_client] outbox capacity reached, drop offline metric"), + {keep_state, State#state{ + outbox = Outbox, + dropped_message_count = State#state.dropped_message_count + 1 + }}; + {error, Reason} -> + logger:warning("[efka_iot_client] append outbox failed, reason: ~p", [Reason]), + {keep_state, State#state{dropped_message_count = State#state.dropped_message_count + 1}} + end + end; + +%% Task的stream流,只做实时的 +handle_event(cast, {task_event_stream, TaskId, Type, Stream}, ?STATE_ACTIVATED, State = #state{socket = Socket}) -> + logger:debug("[efka_iot_client] event_stream task_id: ~p, stream: ~ts", [TaskId, Stream]), + Packet = term_to_binary({<<"message">>, {<<"task_event">>, #{<<"task_id">> => TaskId, <<"type">> => Type, <<"stream">> => Stream}}}), + ok = ssl:send(Socket, Packet), + {keep_state, State}; + +handle_event(cast, {close_task_event_stream, TaskId, Reason}, ?STATE_ACTIVATED, State = #state{socket = Socket}) -> + Packet = term_to_binary({<<"message">>, {<<"task_event">>, #{<<"task_id">> => TaskId, <<"type">> => <<"close">>, <<"stream">> => Reason}}}), + ok = ssl:send(Socket, Packet), + {keep_state, State}; + +%% 其他情况下直接忽略 +handle_event(cast, _, _, State = #state{}) -> + {keep_state, State}; + +handle_event({call, From}, is_activated, ?STATE_ACTIVATED, State = #state{}) -> + {keep_state, State, [{reply, From, true}]}; +handle_event({call, From}, is_activated, _StateName, State = #state{}) -> + {keep_state, State, [{reply, From, false}]}; +handle_event({call, From}, dropped_message_count, _StateName, State = #state{dropped_message_count = DroppedCount}) -> + {keep_state, State, [{reply, From, DroppedCount}]}; + +%% 异步建立到服务器的连接 +handle_event(info, {timeout, _, create_transport}, ?STATE_DISCONNECTED, State) -> + case connect_socket() of + {ok, Socket} -> + Ref = request_ref(), + AuthPacket = auth_packet(Ref), + ok = ssl:send(Socket, AuthPacket), + logger:debug("[efka_iot_client] send auth request, ref: ~p", [Ref]), + {next_state, ?STATE_AUTH, State#state{socket = Socket, auth_ref = Ref}, [{state_timeout, 5000, auth_timeout}]}; + {error, _Reason} -> + schedule_reconnect(), + {keep_state, State#state{socket = undefined}} + end; + +handle_event(state_timeout, auth_timeout, ?STATE_AUTH, State = #state{socket = Socket}) -> + logger:debug("[efka_iot_client] auth request timeout"), + disconnect(Socket), + schedule_reconnect(), + {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined}}; + +handle_event(info, {timeout, TimerRef, ssl_ping}, ?STATE_ACTIVATED, State = #state{socket = Socket, ping_timer_ref = TimerRef}) -> + Packet = term_to_binary({<<"message">>, <<"ping">>}), + case ssl:send(Socket, Packet) of + ok -> + {keep_state, schedule_ssl_ping(State)}; + {error, Reason} -> + logger:warning("[efka_iot_client] send ssl ping failed, reason: ~p", [Reason]), + disconnect(Socket), + schedule_reconnect(), + {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}} + end; +handle_event(info, {timeout, _TimerRef, ssl_ping}, _StateName, State) -> + {keep_state, State}; + +%% 将缓存中的数据推送到服务器端 +handle_event(info, flush_cache, ?STATE_ACTIVATED, State = #state{socket = Socket}) -> + case efka_iot_outbox:next(State#state.outbox) of + {ok, Seq, Packet} -> + ok = ssl:send(Socket, Packet), + case efka_iot_outbox:ack(Seq, State#state.outbox) of + {ok, Outbox} -> + {keep_state, State#state{outbox = Outbox}, [{next_event, info, flush_cache}]}; + {error, Reason} -> + logger:warning("[efka_iot_client] ack outbox failed, seq: ~p, reason: ~p", [Seq, Reason]), + {keep_state, State} + end; + eof -> + {keep_state, State}; + {error, Reason} -> + logger:warning("[efka_iot_client] read outbox failed, reason: ~p", [Reason]), + {keep_state, State} + end; +handle_event(info, flush_cache, _, State) -> + {keep_state, State}; + +%% 处理收到的ssl消息 +handle_event(info, {ssl, Socket, PacketBin}, _, State = #state{socket = Socket}) when is_binary(PacketBin) -> + try binary_to_term(PacketBin, [safe]) of + Packet -> + {keep_state, State, [{next_event, internal, Packet}]} + catch + error:Error -> + logger:warning("[efka_iot_client] binary_to_term get error: ~p, packet_size: ~p", [Error, byte_size(PacketBin)]), + disconnect(Socket), + cancel_ssl_ping(State), + schedule_reconnect(), + {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}} + end; +handle_event(info, {ssl_error, Socket, Reason}, _, State = #state{}) -> + logger:debug("[efka_iot_client] ssl error: ~p", [Reason]), + disconnect(Socket), + cancel_ssl_ping(State), + schedule_reconnect(), + {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}}; +handle_event(info, {ssl_closed, Socket}, _, State = #state{}) -> + logger:debug("[efka_iot_client] ssl closed"), + disconnect(Socket), + cancel_ssl_ping(State), + schedule_reconnect(), + {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined, ping_timer_ref = undefined}}; + +%%% 处理内部消息,ssl收到的消息会先 binary_to_term,再由这里按协议结构模式匹配 + +%% 容器管理命令由 iot 发起,使用 command/command_response 语义。 +handle_event(internal, {<<"command">>, Ref, {<<"container">>, Request}}, ?STATE_ACTIVATED, State = #state{socket = Socket}) -> + handle_container_command(Ref, Request, Socket), + {keep_state, State}; +handle_event(internal, {<<"command">>, Ref, {<<"container">>, Request}}, _StateName, State = #state{socket = Socket}) -> + logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [Request]), + send_container_response(Socket, Ref, {error, <<"agent invalid">>}), + {keep_state, State}; + +%% 处理response +handle_event(internal, {<<"response">>, AuthRef, {<<"auth_response">>, <<"ok">>}}, ?STATE_AUTH, State = #state{auth_ref = AuthRef}) -> + logger:debug("[efka_iot_client] auth success"), + State1 = schedule_ssl_ping(State#state{auth_ref = undefined}), + {next_state, ?STATE_ACTIVATED, State1, [{next_event, info, flush_cache}]}; +handle_event(internal, {<<"response">>, AuthRef, {<<"auth_response">>, {<<"error">>, Reason}}}, ?STATE_AUTH, State = #state{socket = Socket, auth_ref = AuthRef}) -> + logger:debug("[efka_iot_client] auth failed, reason: ~p", [Reason]), + disconnect(Socket), + schedule_reconnect(), + {next_state, ?STATE_DISCONNECTED, State#state{socket = undefined, auth_ref = undefined}}; +handle_event(internal, {<<"response">>, _Ref, Reply}, StateName, State) -> + logger:warning("[efka_iot_client] ignore unexpected response in state ~p: ~p", [StateName, Reply]), + {keep_state, State}; +handle_event(internal, {<<"command_response">>, _Ref, Reply}, StateName, State) -> + logger:warning("[efka_iot_client] ignore unexpected command_response in state ~p: ~p", [StateName, Reply]), + {keep_state, State}; + +%% 处理Pub/Sub机制 +handle_event(internal, {<<"message">>, <<"pong">>}, ?STATE_ACTIVATED, State) -> + {keep_state, State}; +handle_event(internal, {<<"message">>, {<<"pub">>, #{<<"topic">> := Topic, <<"qos">> := Qos, <<"content">> := Content}}}, ?STATE_ACTIVATED, State) -> + logger:debug("[efka_iot_client] get pub topic: ~p, qos: ~p, content: ~p", [Topic, Qos, Content]), + efka_subscription:publish(Topic, Qos, Content), + {keep_state, State}; +handle_event(internal, Packet, _StateName, State) -> + logger:warning("[efka_iot_client] ignore unknown packet: ~p", [Packet]), + {keep_state, State}; + +handle_event(info, Info, _, State = #state{}) -> + logger:notice("[efka_iot_client] get unknown info: ~p", [Info]), + {keep_state, State}. + +-spec handle_container_command(binary(), term(), ssl:sslsocket()) -> ok. +handle_container_command(Ref, #{<<"action">> := <<"list">>}, Socket) -> + Reply = docker_commands:get_containers(), + send_container_response(Socket, Ref, Reply), + ok; +handle_container_command(Ref, #{<<"action">> := <<"deploy">>, <<"task_id">> := TaskId, <<"params">> := Params}, Socket) -> + Reply = docker_deploy_manager:deploy(TaskId, Params), + send_container_response(Socket, Ref, Reply), + ok; +handle_container_command(Ref, #{<<"action">> := <<"start">>, <<"target">> := Target}, Socket) -> + Reply = docker_commands:start_container(container_target(Target)), + send_container_response(Socket, Ref, Reply), + ok; +handle_container_command(Ref, #{<<"action">> := <<"stop">>, <<"target">> := Target, <<"timeout_seconds">> := TimeoutSeconds}, Socket) -> + Reply = docker_commands:stop_container(container_target(Target), TimeoutSeconds), + send_container_response(Socket, Ref, Reply), + ok; +handle_container_command(Ref, #{<<"action">> := <<"kill">>, <<"target">> := Target, <<"signal">> := Signal}, Socket) -> + Reply = docker_commands:kill_container(container_target(Target), to_binary(Signal)), + send_container_response(Socket, Ref, Reply), + ok; +handle_container_command(Ref, #{<<"action">> := <<"remove">>, <<"target">> := Target, <<"force">> := Force, <<"remove_volumes">> := RemoveVolumes}, Socket) -> + Reply = docker_commands:remove_container(container_target(Target), to_bool(Force), to_bool(RemoveVolumes)), + send_container_response(Socket, Ref, Reply), + ok; +handle_container_command(Ref, #{<<"action">> := <<"config">>, <<"target">> := Target, <<"config">> := Config}, Socket) -> + Reply = docker_helper:update_container_config(container_target(Target), iolist_to_binary(Config)), + send_container_response(Socket, Ref, Reply), + ok; +handle_container_command(Ref, Request, Socket) -> + logger:notice("[efka_iot_client] get an invalid command: ~p, agent invalid", [Request]), + send_container_response(Socket, Ref, {error, <<"agent invalid">>}), + ok. + +-spec terminate(term(), atom(), #state{}) -> ok. +terminate(Reason, _StateName, State = #state{socket = Socket, outbox = Outbox}) -> + cancel_ssl_ping(State), + disconnect(Socket), + efka_iot_outbox:close(Outbox), + logger:notice("[efka_iot_client] terminate with reason: ~p", [Reason]), + ok. + +-spec code_change(term(), atom(), #state{}, term()) -> {ok, atom(), #state{}}. +code_change(_OldVsn, StateName, State = #state{}, _Extra) -> + {ok, StateName, State}. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== + +-spec auth_packet(binary()) -> binary(). +auth_packet(Ref) when is_binary(Ref) -> + {ok, AuthInfo} = application:get_env(efka, auth), + UUID = proplists:get_value(uuid, AuthInfo), + Token = proplists:get_value(token, AuthInfo), + + Timestamp = efka_util:timestamp(), + term_to_binary({<<"request">>, Ref, {<<"auth_request">>, #{ + <<"uuid">> => list_to_binary(UUID), + <<"token">> => list_to_binary(Token), + <<"timestamp">> => Timestamp + }}}). + +-spec connect_socket() -> {ok, ssl:sslsocket()} | {error, term()}. +connect_socket() -> + {ok, Props} = application:get_env(efka, iot_server), + Host = proplists:get_value(host, Props), + Port = proplists:get_value(tls_port, Props), + SslOptions = [ + binary, + {active, true}, + {packet, 4}, + {verify, verify_none} + ], + ssl:connect(Host, Port, SslOptions, 5000). + +-spec disconnect(undefined | ssl:sslsocket()) -> ok. +disconnect(undefined) -> + ok; +disconnect(Socket) -> + catch ssl:close(Socket), + ok. + +-spec schedule_reconnect() -> reference(). +schedule_reconnect() -> + erlang:start_timer(5000, self(), create_transport). + +-spec schedule_ssl_ping(#state{}) -> #state{}. +schedule_ssl_ping(State = #state{ping_timer_ref = TimerRef}) -> + cancel_timer(TimerRef), + State#state{ping_timer_ref = erlang:start_timer(?SSL_PING_INTERVAL, self(), ssl_ping)}. + +-spec cancel_ssl_ping(#state{}) -> ok. +cancel_ssl_ping(#state{ping_timer_ref = TimerRef}) -> + cancel_timer(TimerRef). + +-spec cancel_timer(undefined | reference()) -> ok. +cancel_timer(undefined) -> + ok; +cancel_timer(TimerRef) -> + _ = erlang:cancel_timer(TimerRef), + ok. + +-spec request_ref() -> binary(). +request_ref() -> + crypto:strong_rand_bytes(16). + +-spec send_container_response(ssl:sslsocket(), binary(), term()) -> ok. +send_container_response(Socket, Ref, Reply) -> + Packet = term_to_binary({<<"command_response">>, Ref, {<<"container">>, safe_reply(Reply)}}), + ok = ssl:send(Socket, Packet). + +-spec safe_reply(term()) -> term(). +safe_reply(ok) -> + <<"ok">>; +safe_reply({ok, Result}) -> + {<<"ok">>, safe_term(Result)}; +safe_reply({error, Reason}) -> + {<<"error">>, safe_term(Reason)}. + +-spec safe_term(term()) -> term(). +safe_term(true) -> + true; +safe_term(false) -> + false; +safe_term(undefined) -> + undefined; +safe_term(Value) when is_atom(Value) -> + atom_to_binary(Value, utf8); +safe_term(Value) when is_map(Value) -> + maps:from_list([{safe_term(K), safe_term(V)} || {K, V} <- maps:to_list(Value)]); +safe_term(Value) when is_list(Value) -> + [safe_term(Item) || Item <- Value]; +safe_term(Value) when is_tuple(Value) -> + list_to_tuple([safe_term(Item) || Item <- tuple_to_list(Value)]); +safe_term(Value) -> + Value. + +-spec container_target(map()) -> binary(). +container_target(Target) when is_map(Target) -> + NameBin = to_binary(maps:get(<<"name">>, Target, <<>>)), + IdBin = to_binary(maps:get(<<"id">>, Target, <<>>)), + case NameBin of + <<>> -> + true = IdBin =/= <<>>, + IdBin; + _ -> + NameBin + end. + +-spec to_binary(binary() | list()) -> binary(). +to_binary(Value) when is_binary(Value) -> + Value; +to_binary(Value) when is_list(Value) -> + unicode:characters_to_binary(Value). + +-spec to_bool(true | false | 0 | 1) -> boolean(). +to_bool(true) -> + true; +to_bool(1) -> + true; +to_bool(false) -> + false; +to_bool(0) -> + false. diff --git a/apps/efka/src/iot/efka_iot_heartbeat.erl b/apps/efka/src/iot/efka_iot_heartbeat.erl new file mode 100644 index 0000000..ba4076b --- /dev/null +++ b/apps/efka/src/iot/efka_iot_heartbeat.erl @@ -0,0 +1,119 @@ +%%%------------------------------------------------------------------- +%%% @doc UDP heartbeat sender for iot host liveness. +%%% @end +%%%------------------------------------------------------------------- +-module(efka_iot_heartbeat). + +-behaviour(gen_server). + +%% API +-export([start_link/0]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). + +-define(SERVER, ?MODULE). +-define(HEARTBEAT_VERSION, 1). +-define(HEARTBEAT_NONCE_BYTES, 16). +-define(DEFAULT_INTERVAL, 5000). + +-record(state, { + socket :: gen_udp:socket(), + host :: inet:hostname() | inet:ip_address(), + port :: inet:port_number(), + interval :: pos_integer(), + uuid :: binary(), + heartbeat_secret :: binary() +}). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec start_link() -> {ok, pid()} | ignore | {error, term()}. +start_link() -> + gen_server:start_link({local, ?SERVER}, ?MODULE, [], []). + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +-spec init(list()) -> {ok, #state{}} | {stop, term()}. +init([]) -> + ok = application:ensure_started(crypto), + {ok, ServerProps} = application:get_env(efka, iot_server), + {ok, AuthProps} = application:get_env(efka, auth), + HeartbeatProps = case application:get_env(efka, heartbeat) of + {ok, Props} -> Props; + undefined -> [] + end, + Host = proplists:get_value(host, ServerProps), + UdpPort = proplists:get_value(udp_port, ServerProps), + Interval = proplists:get_value(interval, HeartbeatProps, ?DEFAULT_INTERVAL), + UUID = list_to_binary(proplists:get_value(uuid, AuthProps)), + Token = list_to_binary(proplists:get_value(token, AuthProps)), + HeartbeatSecret = crypto:hash(sha256, Token), + case gen_udp:open(0, [binary]) of + {ok, Socket} -> + erlang:send_after(0, self(), heartbeat), + {ok, #state{ + socket = Socket, + host = Host, + port = UdpPort, + interval = Interval, + uuid = UUID, + heartbeat_secret = HeartbeatSecret + }}; + {error, Reason} -> + {stop, Reason} + end. + +-spec handle_call(term(), {pid(), term()}, #state{}) -> {reply, ok, #state{}}. +handle_call(_Request, _From, State) -> + {reply, ok, State}. + +-spec handle_cast(term(), #state{}) -> {noreply, #state{}}. +handle_cast(_Request, State) -> + {noreply, State}. + +-spec handle_info(term(), #state{}) -> {noreply, #state{}}. +handle_info(heartbeat, State = #state{interval = Interval}) -> + ok = send_heartbeat(State), + erlang:send_after(Interval, self(), heartbeat), + {noreply, State}; +handle_info(Info, State) -> + logger:warning("[efka_iot_heartbeat] ignore unknown info: ~p", [Info]), + {noreply, State}. + +-spec terminate(term(), #state{}) -> ok. +terminate(_Reason, #state{socket = Socket}) -> + gen_udp:close(Socket), + ok. + +-spec code_change(term(), #state{}, term()) -> {ok, #state{}}. +code_change(_OldVsn, State, _Extra) -> + {ok, State}. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== + +-spec send_heartbeat(#state{}) -> ok. +send_heartbeat(#state{socket = Socket, host = Host, port = Port, uuid = UUID, heartbeat_secret = HeartbeatSecret}) -> + Packet = heartbeat_packet_iodata(UUID, HeartbeatSecret), + case gen_udp:send(Socket, Host, Port, Packet) of + ok -> + ok; + {error, Reason} -> + logger:warning("[efka_iot_heartbeat] send heartbeat failed, reason: ~p", [Reason]), + ok + end. + +-spec heartbeat_packet_iodata(binary(), binary()) -> iodata(). +heartbeat_packet_iodata(UUID, HeartbeatSecret) when is_binary(UUID), is_binary(HeartbeatSecret) -> + UUIDLen = byte_size(UUID), + Timestamp = efka_util:timestamp(), + Nonce = crypto:strong_rand_bytes(?HEARTBEAT_NONCE_BYTES), + Payload = <>, + Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload), + [Payload, Mac]. diff --git a/apps/efka/src/iot/efka_iot_outbox.erl b/apps/efka/src/iot/efka_iot_outbox.erl new file mode 100644 index 0000000..40a4b83 --- /dev/null +++ b/apps/efka/src/iot/efka_iot_outbox.erl @@ -0,0 +1,662 @@ +%%%------------------------------------------------------------------- +%%% @doc +%%% efka_iot_client 的持久化待发送队列。 +%%% +%%% 本模块用于保存 efka_iot_client 离线期间产生的待发送 packet。底层使用 +%%% 追加写 segment 文件的方式实现。当前场景只有一个写入方和一个读取方,并且 +%%% 读取方每次只读取一条数据。 +%%% +%%% 文件结构: +%%% - `segment-N.log':一个 segment 文件,按写入顺序追加保存 record。 +%%% - `metadata.json':使用原始 JSON 文本保存 write_seq、acked_seq 和 +%%% writer_segment。进程启动时仍然会扫描 segment 文件,并以 segment 文件 +%%% 里的真实数据作为准确信息;因此即使 metadata 落后,也不会导致跳过或删除 +%%% 未消费的数据。 +%%% - outbox 目录通过 open/1 的 dir 参数传入,本模块不直接读取应用环境变量。 +%%% +%%% open/1 参数: +%%% - dir:outbox 数据目录。 +%%% - segment_record_limit:每个 segment 最多保存的 record 数。 +%%% - max_segments:最多保留的 live segment 数。 +%%% - max_record_bytes:允许读取的单条 record 最大 payload 字节数。 +%%% +%%% record 格式: +%%% - 8 字节 magic bytes,当前为 <<"EFKAOBX1">>。 +%%% - 1 字节 version,当前为 1。 +%%% - 1 字节 header size,当前为 22。 +%%% - 4 字节 unsigned big-endian packet size。 +%%% - 8 字节 unsigned big-endian Seq。 +%%% - Packet 原始二进制内容。 +%%% - 4 字节 unsigned big-endian crc32,校验范围为 header + Packet。 +%%% - 读取时会先搜索 magic,再校验 version、header size、packet size 和 +%%% crc32。遇到损坏 record 时,会继续向后寻找下一个合法 magic,尽量恢复 +%%% 后续可读数据。 +%%% +%%% 写入逻辑: +%%% - `append/2' 将一条 record 写入当前 writer segment,随后 fsync 当前 +%%% segment,推进 write_seq,并持久化 metadata。 +%%% - 每个 segment 最多保存多少条数据由 open/1 的 segment_record_limit +%%% 参数决定。 +%%% - outbox 最多保留多少个 live segment 由 open/1 的 max_segments 参数 +%%% 决定。当当前 segment 已满,且 live segment 数已经达到上限时,新 +%%% packet 会被丢弃,已有磁盘数据保持不变。 +%%% - 当当前 segment 已满,但 live segment 数量尚未达到上限时,writer 会 +%%% 关闭旧文件,并开始写入 `segment-(N + 1).log'。 +%%% +%%% 读取逻辑: +%%% - `next/1' 找到第一个 end_seq 大于 acked_seq 的 live segment,打开该 +%%% segment,并只返回第一条 Seq 大于 acked_seq 的 record。 +%%% - 本模块不做批量读取;efka_iot_client 每次只发送并 ack 一条 packet。 +%%% - `ack/2' 接受当前读取到的 Seq。正常情况下 Seq 连续递增;如果某条 +%%% record 损坏并被读取端跳过,ack 允许推进到后续合法 record 的 Seq, +%%% 避免单条坏数据阻断整个 segment 后续数据的消费。 +%%% +%%% segment 清理逻辑: +%%% - 当 `ack/2' 消费完某个 segment 的最后一条 record,并且后续 segment +%%% 里还有未读数据时,已消费的 segment 文件会被删除;下一次读取会从后续 +%%% segment 开始。 +%%% - 当 acked_seq 追上 write_seq 时,说明所有 record 都已经投递完成;此时 +%%% 会删除所有 segment 文件,将计数器重置为 0,下一次 append 从全新的 +%%% `segment-1.log' 的 Seq 1 开始。 +%%% @end +%%%------------------------------------------------------------------- +-module(efka_iot_outbox). + +-export([open/1, close/1, append/2, next/1, ack/2]). +-export_type([outbox/0]). + +-record(segment, { + id :: pos_integer(), + path :: file:filename_all(), + start_seq = 0 :: non_neg_integer(), + end_seq = 0 :: non_neg_integer(), + records = 0 :: non_neg_integer() +}). + +-record(outbox, { + dir :: file:filename_all(), + metadata_path :: file:filename_all(), + writer_segment = 1 :: pos_integer(), + fd :: file:fd(), + segment_record_limit :: pos_integer(), + max_segments :: pos_integer(), + max_record_bytes :: pos_integer(), + segments = [] :: [#segment{}], + next_seq = 1 :: pos_integer(), + write_seq = 0 :: non_neg_integer(), + acked_seq = 0 :: non_neg_integer() +}). + +-type outbox() :: #outbox{}. + +-define(METADATA_FILE, "metadata.json"). +-define(SEGMENT_PREFIX, "segment-"). +-define(SEGMENT_EXT, ".log"). +-define(RECORD_MAGIC, <<"EFKAOBX1">>). +-define(RECORD_MAGIC_SIZE, 8). +-define(RECORD_VERSION, 1). +-define(RECORD_HEADER_SIZE, 22). +-define(RECORD_CRC_SIZE, 4). + +-type open_options() :: #{ + dir := file:filename_all(), + segment_record_limit := pos_integer(), + max_segments := pos_integer(), + max_record_bytes := pos_integer() +}. + +-spec open(open_options()) -> {ok, outbox()} | {error, term()}. +open(Options) when is_map(Options) -> + Dir = maps:get(dir, Options), + SegmentRecordLimit = positive_option(segment_record_limit, Options), + MaxSegments = positive_option(max_segments, Options), + MaxRecordBytes = positive_option(max_record_bytes, Options), + MetadataPath = filename:join(Dir, ?METADATA_FILE), + maybe + ok ?= ensure_dir(Dir), + {_MetaWriteSeq, MetaAckedSeq, MetaWriterSegment} = read_metadata(MetadataPath), + {ok, Segments} ?= load_segments(Dir, MaxRecordBytes), + ScannedWriteSeq = max_segment_end(Segments), + WriteSeq = ScannedWriteSeq, + AckedSeq = min(MetaAckedSeq, WriteSeq), + WriterSegment = writer_segment_id(Segments, MetaWriterSegment), + {ok, Fd} ?= open_writer(segment_path(Dir, WriterSegment)), + Outbox0 = #outbox{ + dir = Dir, + metadata_path = MetadataPath, + writer_segment = WriterSegment, + fd = Fd, + segment_record_limit = SegmentRecordLimit, + max_segments = MaxSegments, + max_record_bytes = MaxRecordBytes, + segments = Segments, + next_seq = WriteSeq + 1, + write_seq = WriteSeq, + acked_seq = AckedSeq + }, + {ok, Outbox1} ?= normalize_open_outbox(Outbox0), + ok ?= persist_metadata(Outbox1), + {ok, Outbox1} + else + {error, Reason} -> + {error, Reason} + end. + +-spec close(outbox()) -> ok. +close(#outbox{fd = Fd}) -> + _ = file:close(Fd), + ok. + +-spec append(binary(), outbox()) -> + {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}. +append(Packet, Outbox0) when is_binary(Packet) -> + case ensure_writable_segment(Outbox0) of + {ok, Outbox1 = #outbox{fd = Fd, next_seq = Seq}} -> + Record = encode_record(Seq, Packet), + maybe + ok ?= file:write(Fd, Record), + ok ?= file:sync(Fd), + Outbox2 = mark_written(Outbox1, Seq), + ok ?= persist_metadata(Outbox2), + {ok, Outbox2} + else + {error, Reason} -> + {error, Reason} + end; + {dropped, capacity_reached, Outbox1} -> + {dropped, capacity_reached, Outbox1}; + {error, Reason} -> + {error, Reason} + end. + +-spec next(outbox()) -> eof | {ok, pos_integer(), binary()} | {error, term()}. +next(#outbox{segments = Segments, acked_seq = AckedSeq, max_record_bytes = MaxRecordBytes}) -> + case next_segment(Segments, AckedSeq) of + undefined -> + eof; + #segment{path = Path} -> + case file:open(Path, [read, binary]) of + {ok, Fd} -> + Result = read_next_unacked(Fd, AckedSeq, MaxRecordBytes), + _ = file:close(Fd), + Result; + {error, enoent} -> + eof; + {error, Reason} -> + {error, Reason} + end + end. + +-spec ack(pos_integer(), outbox()) -> {ok, outbox()} | {error, term()}. +ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq =< AckedSeq -> + {ok, Outbox}; +ack(Seq, Outbox = #outbox{acked_seq = AckedSeq}) when is_integer(Seq), Seq > AckedSeq -> + AckedOutbox = Outbox#outbox{acked_seq = Seq}, + maybe + ok ?= persist_metadata(AckedOutbox), + {ok, NOutbox} ?= maybe_reset_or_prune(AckedOutbox), + ok ?= persist_metadata(NOutbox), + {ok, NOutbox} + else + {error, Reason} -> + {error, Reason} + end; +ack(Seq, #outbox{acked_seq = AckedSeq}) when is_integer(Seq) -> + {error, {invalid_ack, Seq, AckedSeq}}. + +-spec ensure_dir(file:filename_all()) -> ok | {error, term()}. +ensure_dir(Dir) -> + filelib:ensure_dir(filename:join(Dir, "dummy")). + +-spec positive_option(atom(), map()) -> pos_integer(). +positive_option(Key, Options) -> + case maps:get(Key, Options) of + Value when is_integer(Value), Value > 0 -> + Value; + Value -> + error({invalid_option, Key, Value}) + end. + +-spec read_metadata(file:filename_all()) -> + {non_neg_integer(), non_neg_integer(), pos_integer()}. +read_metadata(MetadataPath) -> + case file:read_file(MetadataPath) of + {ok, Bin} -> + decode_metadata(Bin); + {error, _} -> + {0, 0, 1} + end. + +-spec decode_metadata(binary()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}. +decode_metadata(Bin) -> + try json:decode(Bin) of + Metadata -> + safe_metadata(Metadata) + catch + _:_Reason -> + {0, 0, 1} + end. + +-spec safe_metadata(term()) -> {non_neg_integer(), non_neg_integer(), pos_integer()}. +safe_metadata(#{ + <<"write_seq">> := WriteSeq, + <<"acked_seq">> := AckedSeq, + <<"writer_segment">> := WriterSegment +}) + when is_integer(WriteSeq), WriteSeq >= 0, + is_integer(AckedSeq), AckedSeq >= 0, + is_integer(WriterSegment), WriterSegment > 0 -> + {WriteSeq, AckedSeq, WriterSegment}; +safe_metadata(#{<<"write_seq">> := WriteSeq, <<"acked_seq">> := AckedSeq}) + when is_integer(WriteSeq), WriteSeq >= 0, is_integer(AckedSeq), AckedSeq >= 0 -> + {WriteSeq, AckedSeq, 1}; +safe_metadata(_) -> + {0, 0, 1}. + +-spec persist_metadata(outbox()) -> ok | {error, term()}. +persist_metadata(#outbox{ + metadata_path = MetadataPath, + writer_segment = WriterSegment, + write_seq = WriteSeq, + acked_seq = AckedSeq +}) -> + Metadata = iolist_to_binary(json:encode(#{ + <<"write_seq">> => WriteSeq, + <<"acked_seq">> => AckedSeq, + <<"writer_segment">> => WriterSegment + })), + TmpPath = MetadataPath ++ ".tmp", + case file:write_file(TmpPath, Metadata, [write, binary]) of + ok -> + file:rename(TmpPath, MetadataPath); + {error, Reason} -> + {error, Reason} + end. + +-spec load_segments(file:filename_all(), pos_integer()) -> {ok, [#segment{}]} | {error, term()}. +load_segments(Dir, MaxRecordBytes) -> + case file:list_dir(Dir) of + {ok, Names} -> + Ids = lists:sort([Id || Name <- Names, {ok, Id} <- [parse_segment_id(Name)]]), + load_segments(Dir, Ids, MaxRecordBytes, []); + {error, enoent} -> + {ok, []}; + {error, Reason} -> + {error, Reason} + end. + +-spec load_segments(file:filename_all(), [pos_integer()], pos_integer(), [#segment{}]) -> + {ok, [#segment{}]} | {error, term()}. +load_segments(_Dir, [], _MaxRecordBytes, Acc) -> + {ok, lists:reverse(Acc)}; +load_segments(Dir, [Id | Rest], MaxRecordBytes, Acc) -> + Path = segment_path(Dir, Id), + case scan_segment(Path, Id, MaxRecordBytes) of + {ok, undefined} -> + load_segments(Dir, Rest, MaxRecordBytes, Acc); + {ok, Segment} -> + load_segments(Dir, Rest, MaxRecordBytes, [Segment | Acc]); + {error, Reason} -> + {error, Reason} + end. + +-spec parse_segment_id(string()) -> {ok, pos_integer()} | error. +parse_segment_id(Name) -> + case re:run(Name, "^" ++ ?SEGMENT_PREFIX ++ "([0-9]+)\\" ++ ?SEGMENT_EXT ++ "$", + [{capture, [1], list}]) + of + {match, [Digits]} -> + case list_to_integer(Digits) of + Id when Id > 0 -> + {ok, Id}; + _ -> + error + end; + nomatch -> + error + end. + +-spec segment_path(file:filename_all(), pos_integer()) -> file:filename_all(). +segment_path(Dir, Id) -> + filename:join(Dir, ?SEGMENT_PREFIX ++ integer_to_list(Id) ++ ?SEGMENT_EXT). + +-spec open_writer(file:filename_all()) -> {ok, file:fd()} | {error, term()}. +open_writer(Path) -> + file:open(Path, [append, binary]). + +-spec writer_segment_id([#segment{}], pos_integer()) -> pos_integer(). +writer_segment_id([], MetaWriterSegment) -> + max(1, MetaWriterSegment); +writer_segment_id(Segments, _MetaWriterSegment) -> + (lists:last(Segments))#segment.id. + +-spec max_segment_end([#segment{}]) -> non_neg_integer(). +max_segment_end([]) -> + 0; +max_segment_end(Segments) -> + lists:max([Segment#segment.end_seq || Segment <- Segments]). + +-spec normalize_open_outbox(outbox()) -> {ok, outbox()} | {error, term()}. +normalize_open_outbox(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) when Seq > 0 -> + reset_empty_outbox(Outbox); +normalize_open_outbox(Outbox) -> + prune_acked_segments(Outbox). + +-spec ensure_writable_segment(outbox()) -> + {ok, outbox()} | {dropped, capacity_reached, outbox()} | {error, term()}. +ensure_writable_segment(Outbox = #outbox{segments = Segments, writer_segment = WriterSegment}) -> + case find_segment(WriterSegment, Segments) of + undefined -> + {ok, Outbox}; + #segment{records = Records} when Records < Outbox#outbox.segment_record_limit -> + {ok, Outbox}; + #segment{} when length(Segments) >= Outbox#outbox.max_segments -> + {dropped, capacity_reached, Outbox}; + #segment{} -> + rotate_writer(Outbox, WriterSegment + 1) + end. + +-spec rotate_writer(outbox(), pos_integer()) -> {ok, outbox()} | {error, term()}. +rotate_writer(Outbox = #outbox{dir = Dir, fd = Fd}, NewSegment) -> + maybe + ok ?= file:sync(Fd), + {ok, NFd} ?= open_writer(segment_path(Dir, NewSegment)), + _ = file:close(Fd), + {ok, Outbox#outbox{writer_segment = NewSegment, fd = NFd}} + else + {error, Reason} -> + {error, Reason} + end. + +-spec mark_written(outbox(), pos_integer()) -> outbox(). +mark_written(Outbox = #outbox{ + dir = Dir, + writer_segment = WriterSegment, + segments = Segments +}, Seq) -> + Path = segment_path(Dir, WriterSegment), + Segment0 = case find_segment(WriterSegment, Segments) of + undefined -> + #segment{id = WriterSegment, path = Path, start_seq = Seq}; + Segment -> + Segment + end, + Records = Segment0#segment.records + 1, + Segment1 = Segment0#segment{end_seq = Seq, records = Records}, + Outbox#outbox{ + segments = replace_segment(Segment1, Segments), + next_seq = Seq + 1, + write_seq = Seq + }. + +-spec replace_segment(#segment{}, [#segment{}]) -> [#segment{}]. +replace_segment(Segment, Segments) -> + lists:sort( + fun(A, B) -> A#segment.id < B#segment.id end, + [Segment | [S || S <- Segments, S#segment.id =/= Segment#segment.id]] + ). + +-spec find_segment(pos_integer(), [#segment{}]) -> #segment{} | undefined. +find_segment(Id, Segments) -> + case [Segment || Segment <- Segments, Segment#segment.id =:= Id] of + [Segment] -> + Segment; + [] -> + undefined + end. + +-spec next_segment([#segment{}], non_neg_integer()) -> #segment{} | undefined. +next_segment([], _AckedSeq) -> + undefined; +next_segment([Segment = #segment{end_seq = EndSeq} | _Rest], AckedSeq) when EndSeq > AckedSeq -> + Segment; +next_segment([_Segment | Rest], AckedSeq) -> + next_segment(Rest, AckedSeq). + +-spec maybe_reset_or_prune(outbox()) -> {ok, outbox()} | {error, term()}. +maybe_reset_or_prune(Outbox = #outbox{acked_seq = Seq, write_seq = Seq}) -> + reset_empty_outbox(Outbox); +maybe_reset_or_prune(Outbox) -> + prune_acked_segments(Outbox). + +-spec prune_acked_segments(outbox()) -> {ok, outbox()} | {error, term()}. +prune_acked_segments(Outbox = #outbox{segments = Segments, acked_seq = AckedSeq}) -> + {DeleteSegments, KeepSegments} = take_acked_segments(Segments, AckedSeq, []), + maybe + ok ?= delete_files([Segment#segment.path || Segment <- DeleteSegments]), + {ok, Outbox#outbox{segments = KeepSegments}} + else + {error, Reason} -> + {error, Reason} + end. + +-spec take_acked_segments([#segment{}], non_neg_integer(), [#segment{}]) -> + {[#segment{}], [#segment{}]}. +take_acked_segments([Segment = #segment{end_seq = EndSeq} | Rest], AckedSeq, Acc) + when EndSeq =< AckedSeq -> + take_acked_segments(Rest, AckedSeq, [Segment | Acc]); +take_acked_segments(Segments, _AckedSeq, Acc) -> + {lists:reverse(Acc), Segments}. + +-spec reset_empty_outbox(outbox()) -> {ok, outbox()} | {error, term()}. +reset_empty_outbox(Outbox = #outbox{ + dir = Dir, + fd = Fd, + writer_segment = WriterSegment, + segments = Segments +}) -> + CurrentWriterPath = segment_path(Dir, WriterSegment), + SegmentPaths = [Segment#segment.path || Segment <- Segments], + Paths = lists:usort([CurrentWriterPath | SegmentPaths]), + maybe + ok ?= file:close(Fd), + ok ?= delete_files(Paths), + {ok, NFd} ?= open_writer(segment_path(Dir, 1)), + {ok, Outbox#outbox{ + writer_segment = 1, + fd = NFd, + segments = [], + next_seq = 1, + write_seq = 0, + acked_seq = 0 + }} + else + {error, Reason} -> + {error, Reason} + end. + +-spec delete_files([file:filename_all()]) -> ok | {error, term()}. +delete_files([]) -> + ok; +delete_files([Path | Rest]) -> + case file:delete(Path) of + ok -> + delete_files(Rest); + {error, enoent} -> + delete_files(Rest); + {error, Reason} -> + {error, {delete_failed, Path, Reason}} + end. + +-spec scan_segment(file:filename_all(), pos_integer(), pos_integer()) -> + {ok, #segment{} | undefined} | {error, term()}. +scan_segment(Path, Id, MaxRecordBytes) -> + case file:open(Path, [read, binary]) of + {ok, Fd} -> + Result = scan_segment(Fd, Id, Path, MaxRecordBytes, 0, 0, 0), + _ = file:close(Fd), + Result; + {error, enoent} -> + {ok, undefined}; + {error, Reason} -> + {error, Reason} + end. + +-spec scan_segment(file:fd(), pos_integer(), file:filename_all(), pos_integer(), + non_neg_integer(), non_neg_integer(), non_neg_integer()) -> + {ok, #segment{} | undefined} | {error, term()}. +scan_segment(Fd, Id, Path, MaxRecordBytes, StartSeq, EndSeq, Records) -> + case read_record(Fd, MaxRecordBytes) of + eof when Records =:= 0 -> + {ok, undefined}; + eof -> + {ok, #segment{ + id = Id, + path = Path, + start_seq = StartSeq, + end_seq = EndSeq, + records = Records + }}; + {ok, Seq, _Packet} -> + NStartSeq = case StartSeq of + 0 -> Seq; + _ -> StartSeq + end, + scan_segment(Fd, Id, Path, MaxRecordBytes, NStartSeq, Seq, Records + 1); + {error, Reason} -> + {error, Reason} + end. + +-spec encode_record(pos_integer(), binary()) -> binary(). +encode_record(Seq, Packet) -> + PacketSize = byte_size(Packet), + Magic = ?RECORD_MAGIC, + Header = <>, + Crc32 = erlang:crc32(<
>), + <
>. + +-spec read_next_unacked(file:fd(), non_neg_integer(), pos_integer()) -> + eof | {ok, pos_integer(), binary()} | {error, term()}. +read_next_unacked(Fd, AckedSeq, MaxRecordBytes) -> + case read_record(Fd, MaxRecordBytes) of + eof -> + eof; + {ok, Seq, Packet} when Seq > AckedSeq -> + {ok, Seq, Packet}; + {ok, _Seq, _Packet} -> + read_next_unacked(Fd, AckedSeq, MaxRecordBytes); + {error, Reason} -> + {error, Reason} + end. + +-spec read_record(file:fd(), pos_integer()) -> eof | {ok, pos_integer(), binary()} | {error, term()}. +read_record(Fd, MaxRecordBytes) -> + case find_next_magic(Fd) of + eof -> + eof; + {ok, StartPos} -> + case read_record_after_magic(Fd, MaxRecordBytes) of + {ok, Seq, Packet} -> + {ok, Seq, Packet}; + {resync, _Reason} -> + case file:position(Fd, {bof, StartPos + 1}) of + {ok, _} -> + read_record(Fd, MaxRecordBytes); + {error, Reason} -> + {error, Reason} + end + end; + {error, Reason} -> + {error, Reason} + end. + +-spec find_next_magic(file:fd()) -> eof | {ok, non_neg_integer()} | {error, term()}. +find_next_magic(Fd) -> + case file:position(Fd, cur) of + {ok, StartPos} -> + case file:read(Fd, ?RECORD_MAGIC_SIZE) of + eof -> + eof; + {ok, Magic} when byte_size(Magic) < ?RECORD_MAGIC_SIZE -> + eof; + {ok, ?RECORD_MAGIC} -> + {ok, StartPos}; + {ok, Window} -> + find_next_magic(Fd, Window); + {error, Reason} -> + {error, Reason} + end; + {error, Reason} -> + {error, Reason} + end. + +-spec find_next_magic(file:fd(), binary()) -> eof | {ok, non_neg_integer()} | {error, term()}. +find_next_magic(Fd, Window) -> + case file:read(Fd, 1) of + eof -> + eof; + {ok, Byte} -> + Tail = binary:part(Window, 1, ?RECORD_MAGIC_SIZE - 1), + NWindow = <>, + case NWindow of + ?RECORD_MAGIC -> + case file:position(Fd, cur) of + {ok, Pos} -> + {ok, Pos - ?RECORD_MAGIC_SIZE}; + {error, Reason} -> + {error, Reason} + end; + _ -> + find_next_magic(Fd, NWindow) + end; + {error, Reason} -> + {error, Reason} + end. + +-spec read_record_after_magic(file:fd(), pos_integer()) -> + {ok, pos_integer(), binary()} | {resync, term()}. +read_record_after_magic(Fd, MaxRecordBytes) -> + HeaderRestSize = ?RECORD_HEADER_SIZE - ?RECORD_MAGIC_SIZE, + case file:read(Fd, HeaderRestSize) of + {ok, <>} -> + read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, MaxRecordBytes); + {ok, Partial} -> + {resync, {truncated_record_header, Partial}}; + eof -> + {resync, truncated_record_header}; + {error, Reason} -> + {resync, Reason} + end. + +-spec read_record_payload(file:fd(), non_neg_integer(), non_neg_integer(), + non_neg_integer(), non_neg_integer(), pos_integer()) -> + {ok, pos_integer(), binary()} | {resync, term()}. +read_record_payload(_Fd, Version, _HeaderSize, _PacketSize, _Seq, _MaxRecordBytes) + when Version =/= ?RECORD_VERSION -> + {resync, {invalid_record_version, Version}}; +read_record_payload(_Fd, _Version, HeaderSize, _PacketSize, _Seq, _MaxRecordBytes) + when HeaderSize =/= ?RECORD_HEADER_SIZE -> + {resync, {invalid_record_header_size, HeaderSize}}; +read_record_payload(_Fd, _Version, _HeaderSize, PacketSize, _Seq, MaxRecordBytes) + when PacketSize =:= 0; PacketSize > MaxRecordBytes -> + {resync, {invalid_record_packet_size, PacketSize}}; +read_record_payload(_Fd, _Version, _HeaderSize, _PacketSize, Seq, _MaxRecordBytes) + when Seq =:= 0 -> + {resync, {invalid_record_seq, Seq}}; +read_record_payload(Fd, Version, HeaderSize, PacketSize, Seq, _MaxRecordBytes) -> + case file:read(Fd, PacketSize + ?RECORD_CRC_SIZE) of + {ok, <>} -> + validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32); + {ok, Partial} -> + {resync, {truncated_record_payload, PacketSize, byte_size(Partial)}}; + eof -> + {resync, {truncated_record_payload, PacketSize, 0}}; + {error, Reason} -> + {resync, Reason} + end. + +-spec validate_record_crc(non_neg_integer(), non_neg_integer(), non_neg_integer(), + pos_integer(), binary(), non_neg_integer()) -> + {ok, pos_integer(), binary()} | {resync, term()}. +validate_record_crc(Version, HeaderSize, PacketSize, Seq, Packet, Crc32) -> + Magic = ?RECORD_MAGIC, + Header = <>, + case erlang:crc32(<
>) of + Crc32 -> + {ok, Seq, Packet}; + Expected -> + {resync, {crc32_mismatch, Seq, Expected, Crc32}} + end. diff --git a/apps/efka/src/mnesia/cache_model.erl b/apps/efka/src/mnesia/cache_model.erl deleted file mode 100644 index 28fb6f1..0000000 --- a/apps/efka/src/mnesia/cache_model.erl +++ /dev/null @@ -1,74 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author aresei -%%% @copyright (C) 2023, -%%% @doc -%%% -%%% @end -%%% Created : 04. 7月 2023 12:31 -%%%------------------------------------------------------------------- --module(cache_model). --author("aresei"). --include("efka_tables.hrl"). --include_lib("stdlib/include/qlc.hrl"). - --define(TAB, cache). - -%% API --export([create_table/0]). --export([insert/2, get_all_cache/0, fetch_next/0, delete/1, next_id/0]). --export([first_key/0]). - -create_table() -> - %% id生成器 - {atomic, ok} = mnesia:create_table(cache, [ - {attributes, record_info(fields, cache)}, - {record_name, cache}, - {disc_copies, [node()]}, - {type, ordered_set} - ]). - -next_id() -> - id_generator_model:next_id(?TAB). - --spec insert(Method :: integer(), Data :: binary()) -> ok | {error, Reason :: any()}. -insert(Method, Data) when is_integer(Method), is_binary(Data) -> - Cache = #cache{id = next_id(), method = Method, data = Data}, - case mnesia:transaction(fun() -> mnesia:write(?TAB, Cache, write) end) of - {'atomic', ok} -> - ok; - {'aborted', Reason} -> - {error, Reason} - end. - -fetch_next() -> - case mnesia:dirty_first(?TAB) of - '$end_of_table' -> - error; - Id -> - [Entry] = mnesia:dirty_read(?TAB, Id), - {ok, Entry} - end. - -delete(Id) when is_integer(Id) -> - case mnesia:transaction(fun() -> mnesia:delete(?TAB, Id, write) end) of - {'atomic', ok} -> - ok; - {'aborted', Reason} -> - {error, Reason} - end. - --spec get_all_cache() -> [#cache{}]. -get_all_cache() -> - Fun = fun() -> - Q = qlc:q([E || E <- mnesia:table(?TAB)]), - qlc:e(Q) - end, - case mnesia:transaction(Fun) of - {'atomic', Res} -> - Res; - {'aborted', _} -> - [] - end. - -first_key() -> - mnesia:dirty_first(?TAB). diff --git a/apps/efka/src/mnesia/id_generator_model.erl b/apps/efka/src/mnesia/id_generator_model.erl deleted file mode 100644 index 8bc2b80..0000000 --- a/apps/efka/src/mnesia/id_generator_model.erl +++ /dev/null @@ -1,26 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author anlicheng -%%% @copyright (C) 2025, -%%% @doc -%%% -%%% @end -%%% Created : 06. 5月 2025 10:32 -%%%------------------------------------------------------------------- --module(id_generator_model). --author("anlicheng"). --include("efka_tables.hrl"). - -%% API --export([create_table/0, next_id/1]). - -create_table() -> - %% id生成器 - {atomic, ok} = mnesia:create_table(id_generator, [ - {attributes, record_info(fields, id_generator)}, - {record_name, id_generator}, - {disc_copies, [node()]}, - {type, ordered_set} - ]). - -next_id(Tab) when is_atom(Tab) -> - mnesia:dirty_update_counter(id_generator, Tab, 1). \ No newline at end of file diff --git a/apps/efka/src/mnesia/service_model.erl b/apps/efka/src/mnesia/service_model.erl deleted file mode 100644 index d5f3fa7..0000000 --- a/apps/efka/src/mnesia/service_model.erl +++ /dev/null @@ -1,140 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author aresei -%%% @copyright (C) 2023, -%%% @doc -%%% -%%% @end -%%% Created : 04. 7月 2023 12:31 -%%%------------------------------------------------------------------- --module(service_model). --author("aresei"). --include("efka_tables.hrl"). --include_lib("stdlib/include/qlc.hrl"). - --define(TAB, service). - -%% API --export([create_table/0]). --export([insert/1, get_all_services/0, get_all_service_ids/0, get_running_services/0]). --export([get_config_json/1, set_config/2, get_service/1, get_status/1, change_status/2]). --export([display_services/0]). - -create_table() -> - %% id生成器 - {atomic, ok} = mnesia:create_table(service, [ - {attributes, record_info(fields, service)}, - {record_name, service}, - {disc_copies, [node()]}, - {type, ordered_set} - ]). - -insert(Service = #service{}) -> - case mnesia:transaction(fun() -> mnesia:write(?TAB, Service, write) end) of - {'atomic', Res} -> - Res; - {'aborted', Reason} -> - {error, Reason} - end. - -change_status(ServiceId, NewStatus) when is_binary(ServiceId), is_integer(NewStatus) -> - Fun = fun() -> - case mnesia:read(?TAB, ServiceId, write) of - [] -> - mnesia:abort(<<"service not found">>); - [Service] -> - mnesia:write(?TAB, Service#service{status = NewStatus}, write) - end - end, - case mnesia:transaction(Fun) of - {'atomic', ok} -> - ok; - {'aborted', Reason} -> - {error, Reason} - end. - --spec set_config(ServiceId :: binary(), ConfigJson :: binary()) -> ok | {error, Reason :: any()}. -set_config(ServiceId, ConfigJson) when is_binary(ServiceId), is_binary(ConfigJson) -> - Fun = fun() -> - case mnesia:read(?TAB, ServiceId, write) of - [] -> - mnesia:abort(<<"service not found">>); - [S] -> - mnesia:write(?TAB, S#service{config_json = ConfigJson}, write) - end - end, - case mnesia:transaction(Fun) of - {'atomic', ok} -> - ok; - {'aborted', Reason} -> - {error, Reason} - end. - --spec get_config_json(ServiceId :: binary()) -> error | {ok, ConfigJson :: binary()}. -get_config_json(ServiceId) when is_binary(ServiceId) -> - case mnesia:dirty_read(?TAB, ServiceId) of - [] -> - error; - [#service{config_json = ConfigJson}] -> - {ok, ConfigJson} - end. - --spec get_status(ServiceId :: binary()) -> Status :: integer(). -get_status(ServiceId) when is_binary(ServiceId) -> - case mnesia:dirty_read(?TAB, ServiceId) of - [] -> - 0; - [#service{status = Status}] -> - Status - end. - --spec get_service(ServiceId :: binary()) -> error | {ok, Service :: #service{}}. -get_service(ServiceId) when is_binary(ServiceId) -> - case mnesia:dirty_read(?TAB, ServiceId) of - [] -> - error; - [Service] -> - {ok, Service} - end. - --spec get_all_services() -> [#service{}]. -get_all_services() -> - Fun = fun() -> - Q = qlc:q([E || E <- mnesia:table(?TAB)]), - qlc:e(Q) - end, - - case mnesia:transaction(Fun) of - {'atomic', Res} -> - Res; - {'aborted', _} -> - [] - end. - --spec get_all_service_ids() -> [ServiceId :: binary()]. -get_all_service_ids() -> - mnesia:dirty_all_keys(?TAB). - --spec get_running_services() -> {ok, [#service{}]} | {error, Reason :: term()}. -get_running_services() -> - F = fun() -> - Q = qlc:q([E || E <- mnesia:table(?TAB), E#service.status == 1]), - qlc:e(Q) - end, - case mnesia:transaction(F) of - {atomic, Services} -> - {ok, Services}; - {aborted, Error} -> - {error, Error} - end. - -display_services() -> - F = fun() -> - Q = qlc:q([E || E <- mnesia:table(?TAB)]), - qlc:e(Q) - end, - case mnesia:transaction(F) of - {atomic, Services} -> - {ok, Services}; - {aborted, Error} -> - {error, Error} - end. \ No newline at end of file diff --git a/apps/efka/src/mnesia/task_log_model.erl b/apps/efka/src/mnesia/task_log_model.erl deleted file mode 100644 index 617f332..0000000 --- a/apps/efka/src/mnesia/task_log_model.erl +++ /dev/null @@ -1,46 +0,0 @@ -%%%------------------------------------------------------------------- -%%% @author aresei -%%% @copyright (C) 2023, -%%% @doc -%%% -%%% @end -%%% Created : 04. 7月 2023 12:31 -%%%------------------------------------------------------------------- --module(task_log_model). --author("aresei"). --include("efka_tables.hrl"). --include_lib("stdlib/include/qlc.hrl"). - --define(TAB, task_log). - -%% API --export([create_table/0]). --export([insert/2, get_logs/1]). - -create_table() -> - %% id生成器 - {atomic, ok} = mnesia:create_table(task_log, [ - {attributes, record_info(fields, task_log)}, - {record_name, task_log}, - {disc_copies, [node()]}, - {type, ordered_set} - ]). - --spec insert(TaskId :: integer(), Logs :: [binary()]) -> ok | {error, Reason :: term()}. -insert(TaskId, Logs) when is_integer(TaskId), is_list(Logs) -> - TaskLog = #task_log{task_id = TaskId, logs = Logs}, - case mnesia:transaction(fun() -> mnesia:write(?TAB, TaskLog, write) end) of - {'atomic', Res} -> - Res; - {'aborted', Reason} -> - {error, Reason} - end. - --spec get_logs(TaskId :: integer()) -> Logs :: [binary()]. -get_logs(TaskId) when is_integer(TaskId) -> - case mnesia:dirty_read(?TAB, TaskId) of - [] -> - []; - [#task_log{logs = Logs}] -> - Logs - end. \ No newline at end of file diff --git a/apps/efka/src/proto/message_pb.erl b/apps/efka/src/proto/message_pb.erl deleted file mode 100644 index 41a870e..0000000 --- a/apps/efka/src/proto/message_pb.erl +++ /dev/null @@ -1,2564 +0,0 @@ -%% -*- coding: utf-8 -*- -%% % this file is @generated -%% @private -%% Automatically generated, do not edit -%% Generated by gpb_compile version 4.21.1 -%% Version source: git --module(message_pb). - --export([encode_msg/1, encode_msg/2, encode_msg/3]). --export([decode_msg/2, decode_msg/3]). --export([merge_msgs/2, merge_msgs/3, merge_msgs/4]). --export([verify_msg/1, verify_msg/2, verify_msg/3]). --export([get_msg_defs/0]). --export([get_msg_names/0]). --export([get_group_names/0]). --export([get_msg_or_group_names/0]). --export([get_enum_names/0]). --export([find_msg_def/1, fetch_msg_def/1]). --export([find_enum_def/1, fetch_enum_def/1]). --export([enum_symbol_by_value/2, enum_value_by_symbol/2]). --export([get_service_names/0]). --export([get_service_def/1]). --export([get_rpc_names/1]). --export([find_rpc_def/2, fetch_rpc_def/2]). --export([fqbin_to_service_name/1]). --export([service_name_to_fqbin/1]). --export([fqbins_to_service_and_rpc_name/2]). --export([service_and_rpc_name_to_fqbins/2]). --export([fqbin_to_msg_name/1]). --export([msg_name_to_fqbin/1]). --export([fqbin_to_enum_name/1]). --export([enum_name_to_fqbin/1]). --export([get_package_name/0]). --export([uses_packages/0]). --export([source_basename/0]). --export([get_all_source_basenames/0]). --export([get_all_proto_names/0]). --export([get_msg_containment/1]). --export([get_pkg_containment/1]). --export([get_service_containment/1]). --export([get_rpc_containment/1]). --export([get_enum_containment/1]). --export([get_proto_by_msg_name_as_fqbin/1]). --export([get_proto_by_service_name_as_fqbin/1]). --export([get_proto_by_enum_name_as_fqbin/1]). --export([get_protos_by_pkg_name_as_fqbin/1]). --export([gpb_version_as_string/0, gpb_version_as_list/0]). --export([gpb_version_source/0]). - --include("message_pb.hrl"). --include_lib("gpb/include/gpb.hrl"). - -%% enumerated types - --export_type([]). - -%% message types --type auth_request() :: #auth_request{}. - --type auth_reply() :: #auth_reply{}. - --type pub() :: #pub{}. - --type async_call_reply() :: #async_call_reply{}. - --type deploy() :: #deploy{}. - --type fetch_task_log() :: #fetch_task_log{}. - --type invoke() :: #invoke{}. - --type push_service_config() :: #push_service_config{}. - --type data() :: #data{}. - --type ping() :: #ping{}. - --type service_inform() :: #service_inform{}. - --type event() :: #event{}. - --export_type(['auth_request'/0, 'auth_reply'/0, 'pub'/0, 'async_call_reply'/0, 'deploy'/0, 'fetch_task_log'/0, 'invoke'/0, 'push_service_config'/0, 'data'/0, 'ping'/0, 'service_inform'/0, 'event'/0]). --type '$msg_name'() :: auth_request | auth_reply | pub | async_call_reply | deploy | fetch_task_log | invoke | push_service_config | data | ping | service_inform | event. --type '$msg'() :: auth_request() | auth_reply() | pub() | async_call_reply() | deploy() | fetch_task_log() | invoke() | push_service_config() | data() | ping() | service_inform() | event(). --export_type(['$msg_name'/0, '$msg'/0]). - --if(?OTP_RELEASE >= 24). --dialyzer({no_underspecs, encode_msg/1}). --endif. --spec encode_msg('$msg'()) -> binary(). -encode_msg(Msg) when tuple_size(Msg) >= 1 -> encode_msg(Msg, element(1, Msg), []). - --if(?OTP_RELEASE >= 24). --dialyzer({no_underspecs, encode_msg/2}). --endif. --spec encode_msg('$msg'(), '$msg_name'() | list()) -> binary(). -encode_msg(Msg, MsgName) when is_atom(MsgName) -> encode_msg(Msg, MsgName, []); -encode_msg(Msg, Opts) when tuple_size(Msg) >= 1, is_list(Opts) -> encode_msg(Msg, element(1, Msg), Opts). - --if(?OTP_RELEASE >= 24). --dialyzer({no_underspecs, encode_msg/3}). --endif. --spec encode_msg('$msg'(), '$msg_name'(), list()) -> binary(). -encode_msg(Msg, MsgName, Opts) -> - case proplists:get_bool(verify, Opts) of - true -> verify_msg(Msg, MsgName, Opts); - false -> ok - end, - TrUserData = proplists:get_value(user_data, Opts), - case MsgName of - auth_request -> encode_msg_auth_request(id(Msg, TrUserData), TrUserData); - auth_reply -> encode_msg_auth_reply(id(Msg, TrUserData), TrUserData); - pub -> encode_msg_pub(id(Msg, TrUserData), TrUserData); - async_call_reply -> encode_msg_async_call_reply(id(Msg, TrUserData), TrUserData); - deploy -> encode_msg_deploy(id(Msg, TrUserData), TrUserData); - fetch_task_log -> encode_msg_fetch_task_log(id(Msg, TrUserData), TrUserData); - invoke -> encode_msg_invoke(id(Msg, TrUserData), TrUserData); - push_service_config -> encode_msg_push_service_config(id(Msg, TrUserData), TrUserData); - data -> encode_msg_data(id(Msg, TrUserData), TrUserData); - ping -> encode_msg_ping(id(Msg, TrUserData), TrUserData); - service_inform -> encode_msg_service_inform(id(Msg, TrUserData), TrUserData); - event -> encode_msg_event(id(Msg, TrUserData), TrUserData) - end. - - -encode_msg_auth_request(Msg, TrUserData) -> encode_msg_auth_request(Msg, <<>>, TrUserData). - - -encode_msg_auth_request(#auth_request{uuid = F1, username = F2, salt = F3, token = F4, timestamp = F5}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end, - B3 = if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - case is_empty_string(TrF3) of - true -> B2; - false -> e_type_string(TrF3, <>, TrUserData) - end - end - end, - B4 = if F4 == undefined -> B3; - true -> - begin - TrF4 = id(F4, TrUserData), - case is_empty_string(TrF4) of - true -> B3; - false -> e_type_string(TrF4, <>, TrUserData) - end - end - end, - if F5 == undefined -> B4; - true -> - begin - TrF5 = id(F5, TrUserData), - if TrF5 =:= 0 -> B4; - true -> e_varint(TrF5, <>, TrUserData) - end - end - end. - -encode_msg_auth_reply(Msg, TrUserData) -> encode_msg_auth_reply(Msg, <<>>, TrUserData). - - -encode_msg_auth_reply(#auth_reply{code = F1, message = F2}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - if TrF1 =:= 0 -> Bin; - true -> e_varint(TrF1, <>, TrUserData) - end - end - end, - if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end. - -encode_msg_pub(Msg, TrUserData) -> encode_msg_pub(Msg, <<>>, TrUserData). - - -encode_msg_pub(#pub{topic = F1, content = F2}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end. - -encode_msg_async_call_reply(Msg, TrUserData) -> encode_msg_async_call_reply(Msg, <<>>, TrUserData). - - -encode_msg_async_call_reply(#async_call_reply{code = F1, result = F2, message = F3}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - if TrF1 =:= 0 -> Bin; - true -> e_varint(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end, - if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - case is_empty_string(TrF3) of - true -> B2; - false -> e_type_string(TrF3, <>, TrUserData) - end - end - end. - -encode_msg_deploy(Msg, TrUserData) -> encode_msg_deploy(Msg, <<>>, TrUserData). - - -encode_msg_deploy(#deploy{task_id = F1, service_id = F2, tar_url = F3}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - if TrF1 =:= 0 -> Bin; - true -> e_varint(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end, - if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - case is_empty_string(TrF3) of - true -> B2; - false -> e_type_string(TrF3, <>, TrUserData) - end - end - end. - -encode_msg_fetch_task_log(Msg, TrUserData) -> encode_msg_fetch_task_log(Msg, <<>>, TrUserData). - - -encode_msg_fetch_task_log(#fetch_task_log{task_id = F1}, Bin, TrUserData) -> - if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - if TrF1 =:= 0 -> Bin; - true -> e_varint(TrF1, <>, TrUserData) - end - end - end. - -encode_msg_invoke(Msg, TrUserData) -> encode_msg_invoke(Msg, <<>>, TrUserData). - - -encode_msg_invoke(#invoke{service_id = F1, payload = F2, timeout = F3}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end, - if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - if TrF3 =:= 0 -> B2; - true -> e_varint(TrF3, <>, TrUserData) - end - end - end. - -encode_msg_push_service_config(Msg, TrUserData) -> encode_msg_push_service_config(Msg, <<>>, TrUserData). - - -encode_msg_push_service_config(#push_service_config{service_id = F1, config_json = F2, timeout = F3}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end, - if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - if TrF3 =:= 0 -> B2; - true -> e_varint(TrF3, <>, TrUserData) - end - end - end. - -encode_msg_data(Msg, TrUserData) -> encode_msg_data(Msg, <<>>, TrUserData). - - -encode_msg_data(#data{service_id = F1, device_uuid = F2, metric = F3}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end, - if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - case is_empty_string(TrF3) of - true -> B2; - false -> e_type_string(TrF3, <>, TrUserData) - end - end - end. - -encode_msg_ping(Msg, TrUserData) -> encode_msg_ping(Msg, <<>>, TrUserData). - - -encode_msg_ping(#ping{adcode = F1, boot_time = F2, province = F3, city = F4, efka_version = F5, kernel_arch = F6, ips = F7, cpu_core = F8, cpu_load = F9, cpu_temperature = F10, disk = F11, memory = F12, interfaces = F13}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - if TrF2 =:= 0 -> B1; - true -> e_varint(TrF2, <>, TrUserData) - end - end - end, - B3 = if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - case is_empty_string(TrF3) of - true -> B2; - false -> e_type_string(TrF3, <>, TrUserData) - end - end - end, - B4 = if F4 == undefined -> B3; - true -> - begin - TrF4 = id(F4, TrUserData), - case is_empty_string(TrF4) of - true -> B3; - false -> e_type_string(TrF4, <>, TrUserData) - end - end - end, - B5 = if F5 == undefined -> B4; - true -> - begin - TrF5 = id(F5, TrUserData), - case is_empty_string(TrF5) of - true -> B4; - false -> e_type_string(TrF5, <>, TrUserData) - end - end - end, - B6 = if F6 == undefined -> B5; - true -> - begin - TrF6 = id(F6, TrUserData), - case is_empty_string(TrF6) of - true -> B5; - false -> e_type_string(TrF6, <>, TrUserData) - end - end - end, - B7 = begin - TrF7 = id(F7, TrUserData), - if TrF7 == [] -> B6; - true -> e_field_ping_ips(TrF7, B6, TrUserData) - end - end, - B8 = if F8 == undefined -> B7; - true -> - begin - TrF8 = id(F8, TrUserData), - if TrF8 =:= 0 -> B7; - true -> e_varint(TrF8, <>, TrUserData) - end - end - end, - B9 = if F9 == undefined -> B8; - true -> - begin - TrF9 = id(F9, TrUserData), - if TrF9 =:= 0 -> B8; - true -> e_varint(TrF9, <>, TrUserData) - end - end - end, - B10 = if F10 == undefined -> B9; - true -> - begin - TrF10 = id(F10, TrUserData), - if TrF10 =:= +0.0; TrF10 =:= 0 -> B9; - true -> e_type_float(TrF10, <>, TrUserData) - end - end - end, - B11 = begin - TrF11 = id(F11, TrUserData), - if TrF11 == [] -> B10; - true -> e_field_ping_disk(TrF11, B10, TrUserData) - end - end, - B12 = begin - TrF12 = id(F12, TrUserData), - if TrF12 == [] -> B11; - true -> e_field_ping_memory(TrF12, B11, TrUserData) - end - end, - if F13 == undefined -> B12; - true -> - begin - TrF13 = id(F13, TrUserData), - case is_empty_string(TrF13) of - true -> B12; - false -> e_type_string(TrF13, <>, TrUserData) - end - end - end. - -encode_msg_service_inform(Msg, TrUserData) -> encode_msg_service_inform(Msg, <<>>, TrUserData). - - -encode_msg_service_inform(#service_inform{service_id = F1, props = F2, status = F3, timestamp = F4}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - case is_empty_string(TrF2) of - true -> B1; - false -> e_type_string(TrF2, <>, TrUserData) - end - end - end, - B3 = if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - if TrF3 =:= 0 -> B2; - true -> e_varint(TrF3, <>, TrUserData) - end - end - end, - if F4 == undefined -> B3; - true -> - begin - TrF4 = id(F4, TrUserData), - if TrF4 =:= 0 -> B3; - true -> e_varint(TrF4, <>, TrUserData) - end - end - end. - -encode_msg_event(Msg, TrUserData) -> encode_msg_event(Msg, <<>>, TrUserData). - - -encode_msg_event(#event{service_id = F1, event_type = F2, params = F3}, Bin, TrUserData) -> - B1 = if F1 == undefined -> Bin; - true -> - begin - TrF1 = id(F1, TrUserData), - case is_empty_string(TrF1) of - true -> Bin; - false -> e_type_string(TrF1, <>, TrUserData) - end - end - end, - B2 = if F2 == undefined -> B1; - true -> - begin - TrF2 = id(F2, TrUserData), - if TrF2 =:= 0 -> B1; - true -> e_varint(TrF2, <>, TrUserData) - end - end - end, - if F3 == undefined -> B2; - true -> - begin - TrF3 = id(F3, TrUserData), - case is_empty_string(TrF3) of - true -> B2; - false -> e_type_string(TrF3, <>, TrUserData) - end - end - end. - -e_field_ping_ips([Elem | Rest], Bin, TrUserData) -> - Bin2 = <>, - Bin3 = e_type_string(id(Elem, TrUserData), Bin2, TrUserData), - e_field_ping_ips(Rest, Bin3, TrUserData); -e_field_ping_ips([], Bin, _TrUserData) -> Bin. - -e_field_ping_disk(Elems, Bin, TrUserData) when Elems =/= [] -> - SubBin = e_pfield_ping_disk(Elems, <<>>, TrUserData), - Bin2 = <>, - Bin3 = e_varint(byte_size(SubBin), Bin2), - <>; -e_field_ping_disk([], Bin, _TrUserData) -> Bin. - -e_pfield_ping_disk([Value | Rest], Bin, TrUserData) -> - Bin2 = e_type_int32(id(Value, TrUserData), Bin, TrUserData), - e_pfield_ping_disk(Rest, Bin2, TrUserData); -e_pfield_ping_disk([], Bin, _TrUserData) -> Bin. - -e_field_ping_memory(Elems, Bin, TrUserData) when Elems =/= [] -> - SubBin = e_pfield_ping_memory(Elems, <<>>, TrUserData), - Bin2 = <>, - Bin3 = e_varint(byte_size(SubBin), Bin2), - <>; -e_field_ping_memory([], Bin, _TrUserData) -> Bin. - -e_pfield_ping_memory([Value | Rest], Bin, TrUserData) -> - Bin2 = e_type_int32(id(Value, TrUserData), Bin, TrUserData), - e_pfield_ping_memory(Rest, Bin2, TrUserData); -e_pfield_ping_memory([], Bin, _TrUserData) -> Bin. - --compile({nowarn_unused_function,e_type_sint/3}). -e_type_sint(Value, Bin, _TrUserData) when Value >= 0 -> e_varint(Value * 2, Bin); -e_type_sint(Value, Bin, _TrUserData) -> e_varint(Value * -2 - 1, Bin). - --compile({nowarn_unused_function,e_type_int32/3}). -e_type_int32(Value, Bin, _TrUserData) when 0 =< Value, Value =< 127 -> <>; -e_type_int32(Value, Bin, _TrUserData) -> - <> = <>, - e_varint(N, Bin). - --compile({nowarn_unused_function,e_type_int64/3}). -e_type_int64(Value, Bin, _TrUserData) when 0 =< Value, Value =< 127 -> <>; -e_type_int64(Value, Bin, _TrUserData) -> - <> = <>, - e_varint(N, Bin). - --compile({nowarn_unused_function,e_type_bool/3}). -e_type_bool(true, Bin, _TrUserData) -> <>; -e_type_bool(false, Bin, _TrUserData) -> <>; -e_type_bool(1, Bin, _TrUserData) -> <>; -e_type_bool(0, Bin, _TrUserData) -> <>. - --compile({nowarn_unused_function,e_type_string/3}). -e_type_string(S, Bin, _TrUserData) -> - Utf8 = unicode:characters_to_binary(S), - Bin2 = e_varint(byte_size(Utf8), Bin), - <>. - --compile({nowarn_unused_function,e_type_bytes/3}). -e_type_bytes(Bytes, Bin, _TrUserData) when is_binary(Bytes) -> - Bin2 = e_varint(byte_size(Bytes), Bin), - <>; -e_type_bytes(Bytes, Bin, _TrUserData) when is_list(Bytes) -> - BytesBin = iolist_to_binary(Bytes), - Bin2 = e_varint(byte_size(BytesBin), Bin), - <>. - --compile({nowarn_unused_function,e_type_fixed32/3}). -e_type_fixed32(Value, Bin, _TrUserData) -> <>. - --compile({nowarn_unused_function,e_type_sfixed32/3}). -e_type_sfixed32(Value, Bin, _TrUserData) -> <>. - --compile({nowarn_unused_function,e_type_fixed64/3}). -e_type_fixed64(Value, Bin, _TrUserData) -> <>. - --compile({nowarn_unused_function,e_type_sfixed64/3}). -e_type_sfixed64(Value, Bin, _TrUserData) -> <>. - --compile({nowarn_unused_function,e_type_float/3}). -e_type_float(V, Bin, _) when is_number(V) -> <>; -e_type_float(infinity, Bin, _) -> <>; -e_type_float('-infinity', Bin, _) -> <>; -e_type_float(nan, Bin, _) -> <>. - --compile({nowarn_unused_function,e_type_double/3}). -e_type_double(V, Bin, _) when is_number(V) -> <>; -e_type_double(infinity, Bin, _) -> <>; -e_type_double('-infinity', Bin, _) -> <>; -e_type_double(nan, Bin, _) -> <>. - --compile({nowarn_unused_function,e_unknown_elems/2}). -e_unknown_elems([Elem | Rest], Bin) -> - BinR = case Elem of - {varint, FNum, N} -> - BinF = e_varint(FNum bsl 3, Bin), - e_varint(N, BinF); - {length_delimited, FNum, Data} -> - BinF = e_varint(FNum bsl 3 bor 2, Bin), - BinL = e_varint(byte_size(Data), BinF), - <>; - {group, FNum, GroupFields} -> - Bin1 = e_varint(FNum bsl 3 bor 3, Bin), - Bin2 = e_unknown_elems(GroupFields, Bin1), - e_varint(FNum bsl 3 bor 4, Bin2); - {fixed32, FNum, V} -> - BinF = e_varint(FNum bsl 3 bor 5, Bin), - <>; - {fixed64, FNum, V} -> - BinF = e_varint(FNum bsl 3 bor 1, Bin), - <> - end, - e_unknown_elems(Rest, BinR); -e_unknown_elems([], Bin) -> Bin. - --compile({nowarn_unused_function,e_varint/3}). -e_varint(N, Bin, _TrUserData) -> e_varint(N, Bin). - --compile({nowarn_unused_function,e_varint/2}). -e_varint(N, Bin) when N =< 127 -> <>; -e_varint(N, Bin) -> - Bin2 = <>, - e_varint(N bsr 7, Bin2). - -is_empty_string("") -> true; -is_empty_string(<<>>) -> true; -is_empty_string(L) when is_list(L) -> not string_has_chars(L); -is_empty_string(B) when is_binary(B) -> false. - -string_has_chars([C | _]) when is_integer(C) -> true; -string_has_chars([H | T]) -> - case string_has_chars(H) of - true -> true; - false -> string_has_chars(T) - end; -string_has_chars(B) when is_binary(B), byte_size(B) =/= 0 -> true; -string_has_chars(C) when is_integer(C) -> true; -string_has_chars(<<>>) -> false; -string_has_chars([]) -> false. - - -decode_msg(Bin, MsgName) when is_binary(Bin) -> decode_msg(Bin, MsgName, []). - -decode_msg(Bin, MsgName, Opts) when is_binary(Bin) -> - TrUserData = proplists:get_value(user_data, Opts), - decode_msg_1_catch(Bin, MsgName, TrUserData). - --ifdef('OTP_RELEASE'). -decode_msg_1_catch(Bin, MsgName, TrUserData) -> - try decode_msg_2_doit(MsgName, Bin, TrUserData) - catch - error:{gpb_error,_}=Reason:StackTrace -> - erlang:raise(error, Reason, StackTrace); - Class:Reason:StackTrace -> error({gpb_error,{decoding_failure, {Bin, MsgName, {Class, Reason, StackTrace}}}}) - end. --else. -decode_msg_1_catch(Bin, MsgName, TrUserData) -> - try decode_msg_2_doit(MsgName, Bin, TrUserData) - catch - error:{gpb_error,_}=Reason -> - erlang:raise(error, Reason, - erlang:get_stacktrace()); - Class:Reason -> - StackTrace = erlang:get_stacktrace(), - error({gpb_error,{decoding_failure, {Bin, MsgName, {Class, Reason, StackTrace}}}}) - end. --endif. - -decode_msg_2_doit(auth_request, Bin, TrUserData) -> id(decode_msg_auth_request(Bin, TrUserData), TrUserData); -decode_msg_2_doit(auth_reply, Bin, TrUserData) -> id(decode_msg_auth_reply(Bin, TrUserData), TrUserData); -decode_msg_2_doit(pub, Bin, TrUserData) -> id(decode_msg_pub(Bin, TrUserData), TrUserData); -decode_msg_2_doit(async_call_reply, Bin, TrUserData) -> id(decode_msg_async_call_reply(Bin, TrUserData), TrUserData); -decode_msg_2_doit(deploy, Bin, TrUserData) -> id(decode_msg_deploy(Bin, TrUserData), TrUserData); -decode_msg_2_doit(fetch_task_log, Bin, TrUserData) -> id(decode_msg_fetch_task_log(Bin, TrUserData), TrUserData); -decode_msg_2_doit(invoke, Bin, TrUserData) -> id(decode_msg_invoke(Bin, TrUserData), TrUserData); -decode_msg_2_doit(push_service_config, Bin, TrUserData) -> id(decode_msg_push_service_config(Bin, TrUserData), TrUserData); -decode_msg_2_doit(data, Bin, TrUserData) -> id(decode_msg_data(Bin, TrUserData), TrUserData); -decode_msg_2_doit(ping, Bin, TrUserData) -> id(decode_msg_ping(Bin, TrUserData), TrUserData); -decode_msg_2_doit(service_inform, Bin, TrUserData) -> id(decode_msg_service_inform(Bin, TrUserData), TrUserData); -decode_msg_2_doit(event, Bin, TrUserData) -> id(decode_msg_event(Bin, TrUserData), TrUserData). - - - -decode_msg_auth_request(Bin, TrUserData) -> dfp_read_field_def_auth_request(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), id(<<>>, TrUserData), id(<<>>, TrUserData), id(0, TrUserData), TrUserData). - -dfp_read_field_def_auth_request(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> d_field_auth_request_uuid(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -dfp_read_field_def_auth_request(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> d_field_auth_request_username(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -dfp_read_field_def_auth_request(<<34, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> d_field_auth_request_salt(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -dfp_read_field_def_auth_request(<<42, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> d_field_auth_request_token(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -dfp_read_field_def_auth_request(<<48, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> d_field_auth_request_timestamp(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -dfp_read_field_def_auth_request(<<>>, 0, 0, _, F@_1, F@_2, F@_3, F@_4, F@_5, _) -> #auth_request{uuid = F@_1, username = F@_2, salt = F@_3, token = F@_4, timestamp = F@_5}; -dfp_read_field_def_auth_request(Other, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> dg_read_field_def_auth_request(Other, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData). - -dg_read_field_def_auth_request(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) when N < 32 - 7 -> dg_read_field_def_auth_request(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -dg_read_field_def_auth_request(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_auth_request_uuid(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 18 -> d_field_auth_request_username(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 34 -> d_field_auth_request_salt(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 42 -> d_field_auth_request_token(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 48 -> d_field_auth_request_timestamp(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_auth_request(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 1 -> skip_64_auth_request(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 2 -> skip_length_delimited_auth_request(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 3 -> skip_group_auth_request(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); - 5 -> skip_32_auth_request(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) - end - end; -dg_read_field_def_auth_request(<<>>, 0, 0, _, F@_1, F@_2, F@_3, F@_4, F@_5, _) -> #auth_request{uuid = F@_1, username = F@_2, salt = F@_3, token = F@_4, timestamp = F@_5}. - -d_field_auth_request_uuid(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) when N < 57 -> d_field_auth_request_uuid(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -d_field_auth_request_uuid(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, F@_4, F@_5, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_auth_request(RestF, 0, 0, F, NewFValue, F@_2, F@_3, F@_4, F@_5, TrUserData). - -d_field_auth_request_username(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) when N < 57 -> d_field_auth_request_username(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -d_field_auth_request_username(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, F@_4, F@_5, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_auth_request(RestF, 0, 0, F, F@_1, NewFValue, F@_3, F@_4, F@_5, TrUserData). - -d_field_auth_request_salt(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) when N < 57 -> d_field_auth_request_salt(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -d_field_auth_request_salt(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, F@_4, F@_5, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_auth_request(RestF, 0, 0, F, F@_1, F@_2, NewFValue, F@_4, F@_5, TrUserData). - -d_field_auth_request_token(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) when N < 57 -> d_field_auth_request_token(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -d_field_auth_request_token(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, _, F@_5, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_auth_request(RestF, 0, 0, F, F@_1, F@_2, F@_3, NewFValue, F@_5, TrUserData). - -d_field_auth_request_timestamp(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) when N < 57 -> d_field_auth_request_timestamp(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -d_field_auth_request_timestamp(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, _, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_auth_request(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, NewFValue, TrUserData). - -skip_varint_auth_request(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> skip_varint_auth_request(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -skip_varint_auth_request(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> dfp_read_field_def_auth_request(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData). - -skip_length_delimited_auth_request(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) when N < 57 -> skip_length_delimited_auth_request(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData); -skip_length_delimited_auth_request(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_auth_request(Rest2, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData). - -skip_group_auth_request(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_auth_request(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData). - -skip_32_auth_request(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> dfp_read_field_def_auth_request(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData). - -skip_64_auth_request(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData) -> dfp_read_field_def_auth_request(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, TrUserData). - -decode_msg_auth_reply(Bin, TrUserData) -> dfp_read_field_def_auth_reply(Bin, 0, 0, 0, id(0, TrUserData), id(<<>>, TrUserData), TrUserData). - -dfp_read_field_def_auth_reply(<<8, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_auth_reply_code(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); -dfp_read_field_def_auth_reply(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_auth_reply_message(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); -dfp_read_field_def_auth_reply(<<>>, 0, 0, _, F@_1, F@_2, _) -> #auth_reply{code = F@_1, message = F@_2}; -dfp_read_field_def_auth_reply(Other, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dg_read_field_def_auth_reply(Other, Z1, Z2, F, F@_1, F@_2, TrUserData). - -dg_read_field_def_auth_reply(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 32 - 7 -> dg_read_field_def_auth_reply(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -dg_read_field_def_auth_reply(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 8 -> d_field_auth_reply_code(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); - 18 -> d_field_auth_reply_message(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_auth_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 1 -> skip_64_auth_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 2 -> skip_length_delimited_auth_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 3 -> skip_group_auth_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 5 -> skip_32_auth_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData) - end - end; -dg_read_field_def_auth_reply(<<>>, 0, 0, _, F@_1, F@_2, _) -> #auth_reply{code = F@_1, message = F@_2}. - -d_field_auth_reply_code(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_auth_reply_code(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -d_field_auth_reply_code(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_auth_reply(RestF, 0, 0, F, NewFValue, F@_2, TrUserData). - -d_field_auth_reply_message(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_auth_reply_message(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -d_field_auth_reply_message(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_auth_reply(RestF, 0, 0, F, F@_1, NewFValue, TrUserData). - -skip_varint_auth_reply(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> skip_varint_auth_reply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); -skip_varint_auth_reply(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_auth_reply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). - -skip_length_delimited_auth_reply(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> skip_length_delimited_auth_reply(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -skip_length_delimited_auth_reply(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_auth_reply(Rest2, 0, 0, F, F@_1, F@_2, TrUserData). - -skip_group_auth_reply(Bin, _, Z2, FNum, F@_1, F@_2, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_auth_reply(Rest, 0, Z2, FNum, F@_1, F@_2, TrUserData). - -skip_32_auth_reply(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_auth_reply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). - -skip_64_auth_reply(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_auth_reply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). - -decode_msg_pub(Bin, TrUserData) -> dfp_read_field_def_pub(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), TrUserData). - -dfp_read_field_def_pub(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_pub_topic(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); -dfp_read_field_def_pub(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_pub_content(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); -dfp_read_field_def_pub(<<>>, 0, 0, _, F@_1, F@_2, _) -> #pub{topic = F@_1, content = F@_2}; -dfp_read_field_def_pub(Other, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dg_read_field_def_pub(Other, Z1, Z2, F, F@_1, F@_2, TrUserData). - -dg_read_field_def_pub(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 32 - 7 -> dg_read_field_def_pub(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -dg_read_field_def_pub(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_pub_topic(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); - 18 -> d_field_pub_content(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_pub(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 1 -> skip_64_pub(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 2 -> skip_length_delimited_pub(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 3 -> skip_group_pub(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); - 5 -> skip_32_pub(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData) - end - end; -dg_read_field_def_pub(<<>>, 0, 0, _, F@_1, F@_2, _) -> #pub{topic = F@_1, content = F@_2}. - -d_field_pub_topic(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_pub_topic(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -d_field_pub_topic(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_pub(RestF, 0, 0, F, NewFValue, F@_2, TrUserData). - -d_field_pub_content(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_pub_content(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -d_field_pub_content(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_pub(RestF, 0, 0, F, F@_1, NewFValue, TrUserData). - -skip_varint_pub(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> skip_varint_pub(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); -skip_varint_pub(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_pub(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). - -skip_length_delimited_pub(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> skip_length_delimited_pub(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); -skip_length_delimited_pub(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_pub(Rest2, 0, 0, F, F@_1, F@_2, TrUserData). - -skip_group_pub(Bin, _, Z2, FNum, F@_1, F@_2, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_pub(Rest, 0, Z2, FNum, F@_1, F@_2, TrUserData). - -skip_32_pub(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_pub(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). - -skip_64_pub(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_pub(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). - -decode_msg_async_call_reply(Bin, TrUserData) -> dfp_read_field_def_async_call_reply(Bin, 0, 0, 0, id(0, TrUserData), id(<<>>, TrUserData), id(<<>>, TrUserData), TrUserData). - -dfp_read_field_def_async_call_reply(<<8, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_async_call_reply_code(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_async_call_reply(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_async_call_reply_result(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_async_call_reply(<<26, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_async_call_reply_message(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_async_call_reply(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #async_call_reply{code = F@_1, result = F@_2, message = F@_3}; -dfp_read_field_def_async_call_reply(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dg_read_field_def_async_call_reply(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -dg_read_field_def_async_call_reply(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 32 - 7 -> dg_read_field_def_async_call_reply(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -dg_read_field_def_async_call_reply(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 8 -> d_field_async_call_reply_code(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 18 -> d_field_async_call_reply_result(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 26 -> d_field_async_call_reply_message(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_async_call_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 1 -> skip_64_async_call_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 2 -> skip_length_delimited_async_call_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 3 -> skip_group_async_call_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 5 -> skip_32_async_call_reply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData) - end - end; -dg_read_field_def_async_call_reply(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #async_call_reply{code = F@_1, result = F@_2, message = F@_3}. - -d_field_async_call_reply_code(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_async_call_reply_code(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_async_call_reply_code(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_async_call_reply(RestF, 0, 0, F, NewFValue, F@_2, F@_3, TrUserData). - -d_field_async_call_reply_result(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_async_call_reply_result(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_async_call_reply_result(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_async_call_reply(RestF, 0, 0, F, F@_1, NewFValue, F@_3, TrUserData). - -d_field_async_call_reply_message(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_async_call_reply_message(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_async_call_reply_message(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_async_call_reply(RestF, 0, 0, F, F@_1, F@_2, NewFValue, TrUserData). - -skip_varint_async_call_reply(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> skip_varint_async_call_reply(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -skip_varint_async_call_reply(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_async_call_reply(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_length_delimited_async_call_reply(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> skip_length_delimited_async_call_reply(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -skip_length_delimited_async_call_reply(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_async_call_reply(Rest2, 0, 0, F, F@_1, F@_2, F@_3, TrUserData). - -skip_group_async_call_reply(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_async_call_reply(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, TrUserData). - -skip_32_async_call_reply(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_async_call_reply(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_64_async_call_reply(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_async_call_reply(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -decode_msg_deploy(Bin, TrUserData) -> dfp_read_field_def_deploy(Bin, 0, 0, 0, id(0, TrUserData), id(<<>>, TrUserData), id(<<>>, TrUserData), TrUserData). - -dfp_read_field_def_deploy(<<8, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_deploy_task_id(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_deploy(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_deploy_service_id(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_deploy(<<26, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_deploy_tar_url(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_deploy(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #deploy{task_id = F@_1, service_id = F@_2, tar_url = F@_3}; -dfp_read_field_def_deploy(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dg_read_field_def_deploy(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -dg_read_field_def_deploy(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 32 - 7 -> dg_read_field_def_deploy(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -dg_read_field_def_deploy(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 8 -> d_field_deploy_task_id(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 18 -> d_field_deploy_service_id(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 26 -> d_field_deploy_tar_url(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_deploy(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 1 -> skip_64_deploy(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 2 -> skip_length_delimited_deploy(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 3 -> skip_group_deploy(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 5 -> skip_32_deploy(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData) - end - end; -dg_read_field_def_deploy(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #deploy{task_id = F@_1, service_id = F@_2, tar_url = F@_3}. - -d_field_deploy_task_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_deploy_task_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_deploy_task_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_deploy(RestF, 0, 0, F, NewFValue, F@_2, F@_3, TrUserData). - -d_field_deploy_service_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_deploy_service_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_deploy_service_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_deploy(RestF, 0, 0, F, F@_1, NewFValue, F@_3, TrUserData). - -d_field_deploy_tar_url(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_deploy_tar_url(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_deploy_tar_url(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_deploy(RestF, 0, 0, F, F@_1, F@_2, NewFValue, TrUserData). - -skip_varint_deploy(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> skip_varint_deploy(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -skip_varint_deploy(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_deploy(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_length_delimited_deploy(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> skip_length_delimited_deploy(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -skip_length_delimited_deploy(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_deploy(Rest2, 0, 0, F, F@_1, F@_2, F@_3, TrUserData). - -skip_group_deploy(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_deploy(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, TrUserData). - -skip_32_deploy(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_deploy(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_64_deploy(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_deploy(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -decode_msg_fetch_task_log(Bin, TrUserData) -> dfp_read_field_def_fetch_task_log(Bin, 0, 0, 0, id(0, TrUserData), TrUserData). - -dfp_read_field_def_fetch_task_log(<<8, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> d_field_fetch_task_log_task_id(Rest, Z1, Z2, F, F@_1, TrUserData); -dfp_read_field_def_fetch_task_log(<<>>, 0, 0, _, F@_1, _) -> #fetch_task_log{task_id = F@_1}; -dfp_read_field_def_fetch_task_log(Other, Z1, Z2, F, F@_1, TrUserData) -> dg_read_field_def_fetch_task_log(Other, Z1, Z2, F, F@_1, TrUserData). - -dg_read_field_def_fetch_task_log(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 32 - 7 -> dg_read_field_def_fetch_task_log(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); -dg_read_field_def_fetch_task_log(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 8 -> d_field_fetch_task_log_task_id(Rest, 0, 0, 0, F@_1, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_fetch_task_log(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); - 1 -> skip_64_fetch_task_log(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); - 2 -> skip_length_delimited_fetch_task_log(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); - 3 -> skip_group_fetch_task_log(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); - 5 -> skip_32_fetch_task_log(Rest, 0, 0, Key bsr 3, F@_1, TrUserData) - end - end; -dg_read_field_def_fetch_task_log(<<>>, 0, 0, _, F@_1, _) -> #fetch_task_log{task_id = F@_1}. - -d_field_fetch_task_log_task_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> d_field_fetch_task_log_task_id(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); -d_field_fetch_task_log_task_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_fetch_task_log(RestF, 0, 0, F, NewFValue, TrUserData). - -skip_varint_fetch_task_log(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> skip_varint_fetch_task_log(Rest, Z1, Z2, F, F@_1, TrUserData); -skip_varint_fetch_task_log(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> dfp_read_field_def_fetch_task_log(Rest, Z1, Z2, F, F@_1, TrUserData). - -skip_length_delimited_fetch_task_log(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> skip_length_delimited_fetch_task_log(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); -skip_length_delimited_fetch_task_log(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_fetch_task_log(Rest2, 0, 0, F, F@_1, TrUserData). - -skip_group_fetch_task_log(Bin, _, Z2, FNum, F@_1, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_fetch_task_log(Rest, 0, Z2, FNum, F@_1, TrUserData). - -skip_32_fetch_task_log(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> dfp_read_field_def_fetch_task_log(Rest, Z1, Z2, F, F@_1, TrUserData). - -skip_64_fetch_task_log(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> dfp_read_field_def_fetch_task_log(Rest, Z1, Z2, F, F@_1, TrUserData). - -decode_msg_invoke(Bin, TrUserData) -> dfp_read_field_def_invoke(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), id(0, TrUserData), TrUserData). - -dfp_read_field_def_invoke(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_invoke_service_id(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_invoke(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_invoke_payload(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_invoke(<<24, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_invoke_timeout(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_invoke(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #invoke{service_id = F@_1, payload = F@_2, timeout = F@_3}; -dfp_read_field_def_invoke(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dg_read_field_def_invoke(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -dg_read_field_def_invoke(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 32 - 7 -> dg_read_field_def_invoke(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -dg_read_field_def_invoke(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_invoke_service_id(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 18 -> d_field_invoke_payload(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 24 -> d_field_invoke_timeout(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_invoke(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 1 -> skip_64_invoke(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 2 -> skip_length_delimited_invoke(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 3 -> skip_group_invoke(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 5 -> skip_32_invoke(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData) - end - end; -dg_read_field_def_invoke(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #invoke{service_id = F@_1, payload = F@_2, timeout = F@_3}. - -d_field_invoke_service_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_invoke_service_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_invoke_service_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_invoke(RestF, 0, 0, F, NewFValue, F@_2, F@_3, TrUserData). - -d_field_invoke_payload(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_invoke_payload(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_invoke_payload(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_invoke(RestF, 0, 0, F, F@_1, NewFValue, F@_3, TrUserData). - -d_field_invoke_timeout(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_invoke_timeout(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_invoke_timeout(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_invoke(RestF, 0, 0, F, F@_1, F@_2, NewFValue, TrUserData). - -skip_varint_invoke(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> skip_varint_invoke(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -skip_varint_invoke(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_invoke(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_length_delimited_invoke(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> skip_length_delimited_invoke(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -skip_length_delimited_invoke(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_invoke(Rest2, 0, 0, F, F@_1, F@_2, F@_3, TrUserData). - -skip_group_invoke(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_invoke(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, TrUserData). - -skip_32_invoke(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_invoke(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_64_invoke(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_invoke(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -decode_msg_push_service_config(Bin, TrUserData) -> dfp_read_field_def_push_service_config(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), id(0, TrUserData), TrUserData). - -dfp_read_field_def_push_service_config(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_push_service_config_service_id(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_push_service_config(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_push_service_config_config_json(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_push_service_config(<<24, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_push_service_config_timeout(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_push_service_config(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #push_service_config{service_id = F@_1, config_json = F@_2, timeout = F@_3}; -dfp_read_field_def_push_service_config(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dg_read_field_def_push_service_config(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -dg_read_field_def_push_service_config(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 32 - 7 -> dg_read_field_def_push_service_config(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -dg_read_field_def_push_service_config(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_push_service_config_service_id(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 18 -> d_field_push_service_config_config_json(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 24 -> d_field_push_service_config_timeout(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_push_service_config(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 1 -> skip_64_push_service_config(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 2 -> skip_length_delimited_push_service_config(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 3 -> skip_group_push_service_config(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 5 -> skip_32_push_service_config(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData) - end - end; -dg_read_field_def_push_service_config(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #push_service_config{service_id = F@_1, config_json = F@_2, timeout = F@_3}. - -d_field_push_service_config_service_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_push_service_config_service_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_push_service_config_service_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_push_service_config(RestF, 0, 0, F, NewFValue, F@_2, F@_3, TrUserData). - -d_field_push_service_config_config_json(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_push_service_config_config_json(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_push_service_config_config_json(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_push_service_config(RestF, 0, 0, F, F@_1, NewFValue, F@_3, TrUserData). - -d_field_push_service_config_timeout(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_push_service_config_timeout(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_push_service_config_timeout(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_push_service_config(RestF, 0, 0, F, F@_1, F@_2, NewFValue, TrUserData). - -skip_varint_push_service_config(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> skip_varint_push_service_config(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -skip_varint_push_service_config(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_push_service_config(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_length_delimited_push_service_config(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> skip_length_delimited_push_service_config(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -skip_length_delimited_push_service_config(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_push_service_config(Rest2, 0, 0, F, F@_1, F@_2, F@_3, TrUserData). - -skip_group_push_service_config(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_push_service_config(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, TrUserData). - -skip_32_push_service_config(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_push_service_config(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_64_push_service_config(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_push_service_config(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -decode_msg_data(Bin, TrUserData) -> dfp_read_field_def_data(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), id(<<>>, TrUserData), TrUserData). - -dfp_read_field_def_data(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_data_service_id(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_data(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_data_device_uuid(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_data(<<26, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_data_metric(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_data(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #data{service_id = F@_1, device_uuid = F@_2, metric = F@_3}; -dfp_read_field_def_data(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dg_read_field_def_data(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -dg_read_field_def_data(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 32 - 7 -> dg_read_field_def_data(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -dg_read_field_def_data(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_data_service_id(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 18 -> d_field_data_device_uuid(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 26 -> d_field_data_metric(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_data(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 1 -> skip_64_data(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 2 -> skip_length_delimited_data(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 3 -> skip_group_data(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 5 -> skip_32_data(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData) - end - end; -dg_read_field_def_data(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #data{service_id = F@_1, device_uuid = F@_2, metric = F@_3}. - -d_field_data_service_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_data_service_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_data_service_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_data(RestF, 0, 0, F, NewFValue, F@_2, F@_3, TrUserData). - -d_field_data_device_uuid(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_data_device_uuid(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_data_device_uuid(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_data(RestF, 0, 0, F, F@_1, NewFValue, F@_3, TrUserData). - -d_field_data_metric(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_data_metric(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_data_metric(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_data(RestF, 0, 0, F, F@_1, F@_2, NewFValue, TrUserData). - -skip_varint_data(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> skip_varint_data(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -skip_varint_data(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_data(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_length_delimited_data(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> skip_length_delimited_data(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -skip_length_delimited_data(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_data(Rest2, 0, 0, F, F@_1, F@_2, F@_3, TrUserData). - -skip_group_data(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_data(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, TrUserData). - -skip_32_data(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_data(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_64_data(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_data(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -decode_msg_ping(Bin, TrUserData) -> - dfp_read_field_def_ping(Bin, - 0, - 0, - 0, - id(<<>>, TrUserData), - id(0, TrUserData), - id(<<>>, TrUserData), - id(<<>>, TrUserData), - id(<<>>, TrUserData), - id(<<>>, TrUserData), - id([], TrUserData), - id(0, TrUserData), - id(0, TrUserData), - id(0.0, TrUserData), - id([], TrUserData), - id([], TrUserData), - id(<<>>, TrUserData), - TrUserData). - -dfp_read_field_def_ping(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_adcode(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<16, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_boot_time(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<26, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_province(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<34, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_city(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<42, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_efka_version(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<50, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_kernel_arch(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<58, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_ips(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_cpu_core(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<72, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_cpu_load(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<85, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_cpu_temperature(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<90, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_pfield_ping_disk(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<88, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_disk(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<98, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_pfield_ping_memory(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<96, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_memory(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<106, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - d_field_ping_interfaces(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dfp_read_field_def_ping(<<>>, 0, 0, _, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, R1, F@_8, F@_9, F@_10, R2, R3, F@_13, TrUserData) -> - #ping{adcode = F@_1, boot_time = F@_2, province = F@_3, city = F@_4, efka_version = F@_5, kernel_arch = F@_6, ips = lists_reverse(R1, TrUserData), cpu_core = F@_8, cpu_load = F@_9, cpu_temperature = F@_10, disk = lists_reverse(R2, TrUserData), - memory = lists_reverse(R3, TrUserData), interfaces = F@_13}; -dfp_read_field_def_ping(Other, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - dg_read_field_def_ping(Other, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -dg_read_field_def_ping(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 32 - 7 -> - dg_read_field_def_ping(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -dg_read_field_def_ping(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_ping_adcode(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 16 -> d_field_ping_boot_time(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 26 -> d_field_ping_province(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 34 -> d_field_ping_city(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 42 -> d_field_ping_efka_version(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 50 -> d_field_ping_kernel_arch(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 58 -> d_field_ping_ips(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 64 -> d_field_ping_cpu_core(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 72 -> d_field_ping_cpu_load(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 85 -> d_field_ping_cpu_temperature(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 90 -> d_pfield_ping_disk(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 88 -> d_field_ping_disk(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 98 -> d_pfield_ping_memory(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 96 -> d_field_ping_memory(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 106 -> d_field_ping_interfaces(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_ping(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 1 -> skip_64_ping(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 2 -> skip_length_delimited_ping(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 3 -> skip_group_ping(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); - 5 -> skip_32_ping(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) - end - end; -dg_read_field_def_ping(<<>>, 0, 0, _, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, R1, F@_8, F@_9, F@_10, R2, R3, F@_13, TrUserData) -> - #ping{adcode = F@_1, boot_time = F@_2, province = F@_3, city = F@_4, efka_version = F@_5, kernel_arch = F@_6, ips = lists_reverse(R1, TrUserData), cpu_core = F@_8, cpu_load = F@_9, cpu_temperature = F@_10, disk = lists_reverse(R2, TrUserData), - memory = lists_reverse(R3, TrUserData), interfaces = F@_13}. - -d_field_ping_adcode(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_adcode(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_adcode(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_ping(RestF, 0, 0, F, NewFValue, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_boot_time(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_boot_time(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_boot_time(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, NewFValue, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_province(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_province(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_province(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, NewFValue, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_city(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_city(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_city(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, _, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, NewFValue, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_efka_version(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_efka_version(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_efka_version(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, _, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, NewFValue, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_kernel_arch(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_kernel_arch(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_kernel_arch(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, _, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, NewFValue, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_ips(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_ips(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_ips(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, Prev, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, cons(NewFValue, Prev, TrUserData), F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_cpu_core(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_cpu_core(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_cpu_core(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, _, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, NewFValue, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_cpu_load(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_cpu_load(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_cpu_load(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, _, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, NewFValue, F@_10, F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_cpu_temperature(<<0:16, 128, 127, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, _, F@_11, F@_12, F@_13, TrUserData) -> - dfp_read_field_def_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, id(infinity, TrUserData), F@_11, F@_12, F@_13, TrUserData); -d_field_ping_cpu_temperature(<<0:16, 128, 255, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, _, F@_11, F@_12, F@_13, TrUserData) -> - dfp_read_field_def_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, id('-infinity', TrUserData), F@_11, F@_12, F@_13, TrUserData); -d_field_ping_cpu_temperature(<<_:16, 1:1, _:7, _:1, 127:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, _, F@_11, F@_12, F@_13, TrUserData) -> - dfp_read_field_def_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, id(nan, TrUserData), F@_11, F@_12, F@_13, TrUserData); -d_field_ping_cpu_temperature(<>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, _, F@_11, F@_12, F@_13, TrUserData) -> - dfp_read_field_def_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, id(Value, TrUserData), F@_11, F@_12, F@_13, TrUserData). - -d_field_ping_disk(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_disk(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_disk(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, Prev, F@_12, F@_13, TrUserData) -> - {NewFValue, RestF} = {begin <> = <<(X bsl N + Acc):32/unsigned-native>>, id(Res, TrUserData) end, Rest}, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, cons(NewFValue, Prev, TrUserData), F@_12, F@_13, TrUserData). - -d_pfield_ping_disk(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_pfield_ping_disk(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_pfield_ping_disk(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, E, F@_12, F@_13, TrUserData) -> - Len = X bsl N + Acc, - <> = Rest, - NewSeq = d_packed_field_ping_disk(PackedBytes, 0, 0, F, E, TrUserData), - dfp_read_field_def_ping(Rest2, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, NewSeq, F@_12, F@_13, TrUserData). - -d_packed_field_ping_disk(<<1:1, X:7, Rest/binary>>, N, Acc, F, AccSeq, TrUserData) when N < 57 -> d_packed_field_ping_disk(Rest, N + 7, X bsl N + Acc, F, AccSeq, TrUserData); -d_packed_field_ping_disk(<<0:1, X:7, Rest/binary>>, N, Acc, F, AccSeq, TrUserData) -> - {NewFValue, RestF} = {begin <> = <<(X bsl N + Acc):32/unsigned-native>>, id(Res, TrUserData) end, Rest}, - d_packed_field_ping_disk(RestF, 0, 0, F, [NewFValue | AccSeq], TrUserData); -d_packed_field_ping_disk(<<>>, 0, 0, _, AccSeq, _) -> AccSeq. - -d_field_ping_memory(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_memory(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_memory(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, Prev, F@_13, TrUserData) -> - {NewFValue, RestF} = {begin <> = <<(X bsl N + Acc):32/unsigned-native>>, id(Res, TrUserData) end, Rest}, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, cons(NewFValue, Prev, TrUserData), F@_13, TrUserData). - -d_pfield_ping_memory(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_pfield_ping_memory(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_pfield_ping_memory(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, E, F@_13, TrUserData) -> - Len = X bsl N + Acc, - <> = Rest, - NewSeq = d_packed_field_ping_memory(PackedBytes, 0, 0, F, E, TrUserData), - dfp_read_field_def_ping(Rest2, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, NewSeq, F@_13, TrUserData). - -d_packed_field_ping_memory(<<1:1, X:7, Rest/binary>>, N, Acc, F, AccSeq, TrUserData) when N < 57 -> d_packed_field_ping_memory(Rest, N + 7, X bsl N + Acc, F, AccSeq, TrUserData); -d_packed_field_ping_memory(<<0:1, X:7, Rest/binary>>, N, Acc, F, AccSeq, TrUserData) -> - {NewFValue, RestF} = {begin <> = <<(X bsl N + Acc):32/unsigned-native>>, id(Res, TrUserData) end, Rest}, - d_packed_field_ping_memory(RestF, 0, 0, F, [NewFValue | AccSeq], TrUserData); -d_packed_field_ping_memory(<<>>, 0, 0, _, AccSeq, _) -> AccSeq. - -d_field_ping_interfaces(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - d_field_ping_interfaces(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -d_field_ping_interfaces(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, _, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_ping(RestF, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, NewFValue, TrUserData). - -skip_varint_ping(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - skip_varint_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -skip_varint_ping(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - dfp_read_field_def_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -skip_length_delimited_ping(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) when N < 57 -> - skip_length_delimited_ping(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData); -skip_length_delimited_ping(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_ping(Rest2, 0, 0, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -skip_group_ping(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_ping(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -skip_32_ping(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - dfp_read_field_def_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -skip_64_ping(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData) -> - dfp_read_field_def_ping(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, F@_5, F@_6, F@_7, F@_8, F@_9, F@_10, F@_11, F@_12, F@_13, TrUserData). - -decode_msg_service_inform(Bin, TrUserData) -> dfp_read_field_def_service_inform(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), id(0, TrUserData), id(0, TrUserData), TrUserData). - -dfp_read_field_def_service_inform(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> d_field_service_inform_service_id(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -dfp_read_field_def_service_inform(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> d_field_service_inform_props(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -dfp_read_field_def_service_inform(<<24, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> d_field_service_inform_status(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -dfp_read_field_def_service_inform(<<32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> d_field_service_inform_timestamp(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -dfp_read_field_def_service_inform(<<>>, 0, 0, _, F@_1, F@_2, F@_3, F@_4, _) -> #service_inform{service_id = F@_1, props = F@_2, status = F@_3, timestamp = F@_4}; -dfp_read_field_def_service_inform(Other, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> dg_read_field_def_service_inform(Other, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData). - -dg_read_field_def_service_inform(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData) when N < 32 - 7 -> dg_read_field_def_service_inform(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -dg_read_field_def_service_inform(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, F@_4, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_service_inform_service_id(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, TrUserData); - 18 -> d_field_service_inform_props(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, TrUserData); - 24 -> d_field_service_inform_status(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, TrUserData); - 32 -> d_field_service_inform_timestamp(Rest, 0, 0, 0, F@_1, F@_2, F@_3, F@_4, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_service_inform(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, TrUserData); - 1 -> skip_64_service_inform(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, TrUserData); - 2 -> skip_length_delimited_service_inform(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, TrUserData); - 3 -> skip_group_service_inform(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, TrUserData); - 5 -> skip_32_service_inform(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, F@_4, TrUserData) - end - end; -dg_read_field_def_service_inform(<<>>, 0, 0, _, F@_1, F@_2, F@_3, F@_4, _) -> #service_inform{service_id = F@_1, props = F@_2, status = F@_3, timestamp = F@_4}. - -d_field_service_inform_service_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData) when N < 57 -> d_field_service_inform_service_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -d_field_service_inform_service_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, F@_4, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_service_inform(RestF, 0, 0, F, NewFValue, F@_2, F@_3, F@_4, TrUserData). - -d_field_service_inform_props(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData) when N < 57 -> d_field_service_inform_props(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -d_field_service_inform_props(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, F@_4, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_service_inform(RestF, 0, 0, F, F@_1, NewFValue, F@_3, F@_4, TrUserData). - -d_field_service_inform_status(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData) when N < 57 -> d_field_service_inform_status(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -d_field_service_inform_status(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, F@_4, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_service_inform(RestF, 0, 0, F, F@_1, F@_2, NewFValue, F@_4, TrUserData). - -d_field_service_inform_timestamp(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData) when N < 57 -> d_field_service_inform_timestamp(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -d_field_service_inform_timestamp(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, _, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_service_inform(RestF, 0, 0, F, F@_1, F@_2, F@_3, NewFValue, TrUserData). - -skip_varint_service_inform(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> skip_varint_service_inform(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -skip_varint_service_inform(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> dfp_read_field_def_service_inform(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData). - -skip_length_delimited_service_inform(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData) when N < 57 -> skip_length_delimited_service_inform(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData); -skip_length_delimited_service_inform(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_service_inform(Rest2, 0, 0, F, F@_1, F@_2, F@_3, F@_4, TrUserData). - -skip_group_service_inform(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, F@_4, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_service_inform(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, F@_4, TrUserData). - -skip_32_service_inform(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> dfp_read_field_def_service_inform(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData). - -skip_64_service_inform(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData) -> dfp_read_field_def_service_inform(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, F@_4, TrUserData). - -decode_msg_event(Bin, TrUserData) -> dfp_read_field_def_event(Bin, 0, 0, 0, id(<<>>, TrUserData), id(0, TrUserData), id(<<>>, TrUserData), TrUserData). - -dfp_read_field_def_event(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_event_service_id(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_event(<<16, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_event_event_type(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_event(<<26, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> d_field_event_params(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -dfp_read_field_def_event(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #event{service_id = F@_1, event_type = F@_2, params = F@_3}; -dfp_read_field_def_event(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dg_read_field_def_event(Other, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -dg_read_field_def_event(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 32 - 7 -> dg_read_field_def_event(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -dg_read_field_def_event(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, F@_3, TrUserData) -> - Key = X bsl N + Acc, - case Key of - 10 -> d_field_event_service_id(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 16 -> d_field_event_event_type(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - 26 -> d_field_event_params(Rest, 0, 0, 0, F@_1, F@_2, F@_3, TrUserData); - _ -> - case Key band 7 of - 0 -> skip_varint_event(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 1 -> skip_64_event(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 2 -> skip_length_delimited_event(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 3 -> skip_group_event(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData); - 5 -> skip_32_event(Rest, 0, 0, Key bsr 3, F@_1, F@_2, F@_3, TrUserData) - end - end; -dg_read_field_def_event(<<>>, 0, 0, _, F@_1, F@_2, F@_3, _) -> #event{service_id = F@_1, event_type = F@_2, params = F@_3}. - -d_field_event_service_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_event_service_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_event_service_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, F@_3, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_event(RestF, 0, 0, F, NewFValue, F@_2, F@_3, TrUserData). - -d_field_event_event_type(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_event_event_type(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_event_event_type(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, F@_3, TrUserData) -> - {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, - dfp_read_field_def_event(RestF, 0, 0, F, F@_1, NewFValue, F@_3, TrUserData). - -d_field_event_params(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> d_field_event_params(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -d_field_event_params(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, _, TrUserData) -> - {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, - dfp_read_field_def_event(RestF, 0, 0, F, F@_1, F@_2, NewFValue, TrUserData). - -skip_varint_event(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> skip_varint_event(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData); -skip_varint_event(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_event(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_length_delimited_event(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) when N < 57 -> skip_length_delimited_event(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, F@_3, TrUserData); -skip_length_delimited_event(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, F@_3, TrUserData) -> - Length = X bsl N + Acc, - <<_:Length/binary, Rest2/binary>> = Rest, - dfp_read_field_def_event(Rest2, 0, 0, F, F@_1, F@_2, F@_3, TrUserData). - -skip_group_event(Bin, _, Z2, FNum, F@_1, F@_2, F@_3, TrUserData) -> - {_, Rest} = read_group(Bin, FNum), - dfp_read_field_def_event(Rest, 0, Z2, FNum, F@_1, F@_2, F@_3, TrUserData). - -skip_32_event(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_event(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -skip_64_event(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData) -> dfp_read_field_def_event(Rest, Z1, Z2, F, F@_1, F@_2, F@_3, TrUserData). - -read_group(Bin, FieldNum) -> - {NumBytes, EndTagLen} = read_gr_b(Bin, 0, 0, 0, 0, FieldNum), - <> = Bin, - {Group, Rest}. - -%% Like skipping over fields, but record the total length, -%% Each field is <(FieldNum bsl 3) bor FieldType> ++ -%% Record the length because varints may be non-optimally encoded. -%% -%% Groups can be nested, but assume the same FieldNum cannot be nested -%% because group field numbers are shared with the rest of the fields -%% numbers. Thus we can search just for an group-end with the same -%% field number. -%% -%% (The only time the same group field number could occur would -%% be in a nested sub message, but then it would be inside a -%% length-delimited entry, which we skip-read by length.) -read_gr_b(<<1:1, X:7, Tl/binary>>, N, Acc, NumBytes, TagLen, FieldNum) - when N < (32-7) -> - read_gr_b(Tl, N+7, X bsl N + Acc, NumBytes, TagLen+1, FieldNum); -read_gr_b(<<0:1, X:7, Tl/binary>>, N, Acc, NumBytes, TagLen, - FieldNum) -> - Key = X bsl N + Acc, - TagLen1 = TagLen + 1, - case {Key bsr 3, Key band 7} of - {FieldNum, 4} -> % 4 = group_end - {NumBytes, TagLen1}; - {_, 0} -> % 0 = varint - read_gr_vi(Tl, 0, NumBytes + TagLen1, FieldNum); - {_, 1} -> % 1 = bits64 - <<_:64, Tl2/binary>> = Tl, - read_gr_b(Tl2, 0, 0, NumBytes + TagLen1 + 8, 0, FieldNum); - {_, 2} -> % 2 = length_delimited - read_gr_ld(Tl, 0, 0, NumBytes + TagLen1, FieldNum); - {_, 3} -> % 3 = group_start - read_gr_b(Tl, 0, 0, NumBytes + TagLen1, 0, FieldNum); - {_, 4} -> % 4 = group_end - read_gr_b(Tl, 0, 0, NumBytes + TagLen1, 0, FieldNum); - {_, 5} -> % 5 = bits32 - <<_:32, Tl2/binary>> = Tl, - read_gr_b(Tl2, 0, 0, NumBytes + TagLen1 + 4, 0, FieldNum) - end. - -read_gr_vi(<<1:1, _:7, Tl/binary>>, N, NumBytes, FieldNum) - when N < (64-7) -> - read_gr_vi(Tl, N+7, NumBytes+1, FieldNum); -read_gr_vi(<<0:1, _:7, Tl/binary>>, _, NumBytes, FieldNum) -> - read_gr_b(Tl, 0, 0, NumBytes+1, 0, FieldNum). - -read_gr_ld(<<1:1, X:7, Tl/binary>>, N, Acc, NumBytes, FieldNum) - when N < (64-7) -> - read_gr_ld(Tl, N+7, X bsl N + Acc, NumBytes+1, FieldNum); -read_gr_ld(<<0:1, X:7, Tl/binary>>, N, Acc, NumBytes, FieldNum) -> - Len = X bsl N + Acc, - NumBytes1 = NumBytes + 1, - <<_:Len/binary, Tl2/binary>> = Tl, - read_gr_b(Tl2, 0, 0, NumBytes1 + Len, 0, FieldNum). - -merge_msgs(Prev, New) when element(1, Prev) =:= element(1, New) -> merge_msgs(Prev, New, element(1, Prev), []). - -merge_msgs(Prev, New, MsgName) when is_atom(MsgName) -> merge_msgs(Prev, New, MsgName, []); -merge_msgs(Prev, New, Opts) when element(1, Prev) =:= element(1, New), is_list(Opts) -> merge_msgs(Prev, New, element(1, Prev), Opts). - -merge_msgs(Prev, New, MsgName, Opts) -> - TrUserData = proplists:get_value(user_data, Opts), - case MsgName of - auth_request -> merge_msg_auth_request(Prev, New, TrUserData); - auth_reply -> merge_msg_auth_reply(Prev, New, TrUserData); - pub -> merge_msg_pub(Prev, New, TrUserData); - async_call_reply -> merge_msg_async_call_reply(Prev, New, TrUserData); - deploy -> merge_msg_deploy(Prev, New, TrUserData); - fetch_task_log -> merge_msg_fetch_task_log(Prev, New, TrUserData); - invoke -> merge_msg_invoke(Prev, New, TrUserData); - push_service_config -> merge_msg_push_service_config(Prev, New, TrUserData); - data -> merge_msg_data(Prev, New, TrUserData); - ping -> merge_msg_ping(Prev, New, TrUserData); - service_inform -> merge_msg_service_inform(Prev, New, TrUserData); - event -> merge_msg_event(Prev, New, TrUserData) - end. - --compile({nowarn_unused_function,merge_msg_auth_request/3}). -merge_msg_auth_request(#auth_request{uuid = PFuuid, username = PFusername, salt = PFsalt, token = PFtoken, timestamp = PFtimestamp}, #auth_request{uuid = NFuuid, username = NFusername, salt = NFsalt, token = NFtoken, timestamp = NFtimestamp}, _) -> - #auth_request{uuid = - if NFuuid =:= undefined -> PFuuid; - true -> NFuuid - end, - username = - if NFusername =:= undefined -> PFusername; - true -> NFusername - end, - salt = - if NFsalt =:= undefined -> PFsalt; - true -> NFsalt - end, - token = - if NFtoken =:= undefined -> PFtoken; - true -> NFtoken - end, - timestamp = - if NFtimestamp =:= undefined -> PFtimestamp; - true -> NFtimestamp - end}. - --compile({nowarn_unused_function,merge_msg_auth_reply/3}). -merge_msg_auth_reply(#auth_reply{code = PFcode, message = PFmessage}, #auth_reply{code = NFcode, message = NFmessage}, _) -> - #auth_reply{code = - if NFcode =:= undefined -> PFcode; - true -> NFcode - end, - message = - if NFmessage =:= undefined -> PFmessage; - true -> NFmessage - end}. - --compile({nowarn_unused_function,merge_msg_pub/3}). -merge_msg_pub(#pub{topic = PFtopic, content = PFcontent}, #pub{topic = NFtopic, content = NFcontent}, _) -> - #pub{topic = - if NFtopic =:= undefined -> PFtopic; - true -> NFtopic - end, - content = - if NFcontent =:= undefined -> PFcontent; - true -> NFcontent - end}. - --compile({nowarn_unused_function,merge_msg_async_call_reply/3}). -merge_msg_async_call_reply(#async_call_reply{code = PFcode, result = PFresult, message = PFmessage}, #async_call_reply{code = NFcode, result = NFresult, message = NFmessage}, _) -> - #async_call_reply{code = - if NFcode =:= undefined -> PFcode; - true -> NFcode - end, - result = - if NFresult =:= undefined -> PFresult; - true -> NFresult - end, - message = - if NFmessage =:= undefined -> PFmessage; - true -> NFmessage - end}. - --compile({nowarn_unused_function,merge_msg_deploy/3}). -merge_msg_deploy(#deploy{task_id = PFtask_id, service_id = PFservice_id, tar_url = PFtar_url}, #deploy{task_id = NFtask_id, service_id = NFservice_id, tar_url = NFtar_url}, _) -> - #deploy{task_id = - if NFtask_id =:= undefined -> PFtask_id; - true -> NFtask_id - end, - service_id = - if NFservice_id =:= undefined -> PFservice_id; - true -> NFservice_id - end, - tar_url = - if NFtar_url =:= undefined -> PFtar_url; - true -> NFtar_url - end}. - --compile({nowarn_unused_function,merge_msg_fetch_task_log/3}). -merge_msg_fetch_task_log(#fetch_task_log{task_id = PFtask_id}, #fetch_task_log{task_id = NFtask_id}, _) -> - #fetch_task_log{task_id = - if NFtask_id =:= undefined -> PFtask_id; - true -> NFtask_id - end}. - --compile({nowarn_unused_function,merge_msg_invoke/3}). -merge_msg_invoke(#invoke{service_id = PFservice_id, payload = PFpayload, timeout = PFtimeout}, #invoke{service_id = NFservice_id, payload = NFpayload, timeout = NFtimeout}, _) -> - #invoke{service_id = - if NFservice_id =:= undefined -> PFservice_id; - true -> NFservice_id - end, - payload = - if NFpayload =:= undefined -> PFpayload; - true -> NFpayload - end, - timeout = - if NFtimeout =:= undefined -> PFtimeout; - true -> NFtimeout - end}. - --compile({nowarn_unused_function,merge_msg_push_service_config/3}). -merge_msg_push_service_config(#push_service_config{service_id = PFservice_id, config_json = PFconfig_json, timeout = PFtimeout}, #push_service_config{service_id = NFservice_id, config_json = NFconfig_json, timeout = NFtimeout}, _) -> - #push_service_config{service_id = - if NFservice_id =:= undefined -> PFservice_id; - true -> NFservice_id - end, - config_json = - if NFconfig_json =:= undefined -> PFconfig_json; - true -> NFconfig_json - end, - timeout = - if NFtimeout =:= undefined -> PFtimeout; - true -> NFtimeout - end}. - --compile({nowarn_unused_function,merge_msg_data/3}). -merge_msg_data(#data{service_id = PFservice_id, device_uuid = PFdevice_uuid, metric = PFmetric}, #data{service_id = NFservice_id, device_uuid = NFdevice_uuid, metric = NFmetric}, _) -> - #data{service_id = - if NFservice_id =:= undefined -> PFservice_id; - true -> NFservice_id - end, - device_uuid = - if NFdevice_uuid =:= undefined -> PFdevice_uuid; - true -> NFdevice_uuid - end, - metric = - if NFmetric =:= undefined -> PFmetric; - true -> NFmetric - end}. - --compile({nowarn_unused_function,merge_msg_ping/3}). -merge_msg_ping(#ping{adcode = PFadcode, boot_time = PFboot_time, province = PFprovince, city = PFcity, efka_version = PFefka_version, kernel_arch = PFkernel_arch, ips = PFips, cpu_core = PFcpu_core, cpu_load = PFcpu_load, - cpu_temperature = PFcpu_temperature, disk = PFdisk, memory = PFmemory, interfaces = PFinterfaces}, - #ping{adcode = NFadcode, boot_time = NFboot_time, province = NFprovince, city = NFcity, efka_version = NFefka_version, kernel_arch = NFkernel_arch, ips = NFips, cpu_core = NFcpu_core, cpu_load = NFcpu_load, cpu_temperature = NFcpu_temperature, - disk = NFdisk, memory = NFmemory, interfaces = NFinterfaces}, - TrUserData) -> - #ping{adcode = - if NFadcode =:= undefined -> PFadcode; - true -> NFadcode - end, - boot_time = - if NFboot_time =:= undefined -> PFboot_time; - true -> NFboot_time - end, - province = - if NFprovince =:= undefined -> PFprovince; - true -> NFprovince - end, - city = - if NFcity =:= undefined -> PFcity; - true -> NFcity - end, - efka_version = - if NFefka_version =:= undefined -> PFefka_version; - true -> NFefka_version - end, - kernel_arch = - if NFkernel_arch =:= undefined -> PFkernel_arch; - true -> NFkernel_arch - end, - ips = - if PFips /= undefined, NFips /= undefined -> 'erlang_++'(PFips, NFips, TrUserData); - PFips == undefined -> NFips; - NFips == undefined -> PFips - end, - cpu_core = - if NFcpu_core =:= undefined -> PFcpu_core; - true -> NFcpu_core - end, - cpu_load = - if NFcpu_load =:= undefined -> PFcpu_load; - true -> NFcpu_load - end, - cpu_temperature = - if NFcpu_temperature =:= undefined -> PFcpu_temperature; - true -> NFcpu_temperature - end, - disk = - if PFdisk /= undefined, NFdisk /= undefined -> 'erlang_++'(PFdisk, NFdisk, TrUserData); - PFdisk == undefined -> NFdisk; - NFdisk == undefined -> PFdisk - end, - memory = - if PFmemory /= undefined, NFmemory /= undefined -> 'erlang_++'(PFmemory, NFmemory, TrUserData); - PFmemory == undefined -> NFmemory; - NFmemory == undefined -> PFmemory - end, - interfaces = - if NFinterfaces =:= undefined -> PFinterfaces; - true -> NFinterfaces - end}. - --compile({nowarn_unused_function,merge_msg_service_inform/3}). -merge_msg_service_inform(#service_inform{service_id = PFservice_id, props = PFprops, status = PFstatus, timestamp = PFtimestamp}, #service_inform{service_id = NFservice_id, props = NFprops, status = NFstatus, timestamp = NFtimestamp}, _) -> - #service_inform{service_id = - if NFservice_id =:= undefined -> PFservice_id; - true -> NFservice_id - end, - props = - if NFprops =:= undefined -> PFprops; - true -> NFprops - end, - status = - if NFstatus =:= undefined -> PFstatus; - true -> NFstatus - end, - timestamp = - if NFtimestamp =:= undefined -> PFtimestamp; - true -> NFtimestamp - end}. - --compile({nowarn_unused_function,merge_msg_event/3}). -merge_msg_event(#event{service_id = PFservice_id, event_type = PFevent_type, params = PFparams}, #event{service_id = NFservice_id, event_type = NFevent_type, params = NFparams}, _) -> - #event{service_id = - if NFservice_id =:= undefined -> PFservice_id; - true -> NFservice_id - end, - event_type = - if NFevent_type =:= undefined -> PFevent_type; - true -> NFevent_type - end, - params = - if NFparams =:= undefined -> PFparams; - true -> NFparams - end}. - - -verify_msg(Msg) when tuple_size(Msg) >= 1 -> verify_msg(Msg, element(1, Msg), []); -verify_msg(X) -> mk_type_error(not_a_known_message, X, []). - -verify_msg(Msg, MsgName) when is_atom(MsgName) -> verify_msg(Msg, MsgName, []); -verify_msg(Msg, Opts) when tuple_size(Msg) >= 1 -> verify_msg(Msg, element(1, Msg), Opts); -verify_msg(X, _Opts) -> mk_type_error(not_a_known_message, X, []). - -verify_msg(Msg, MsgName, Opts) -> - TrUserData = proplists:get_value(user_data, Opts), - case MsgName of - auth_request -> v_msg_auth_request(Msg, [MsgName], TrUserData); - auth_reply -> v_msg_auth_reply(Msg, [MsgName], TrUserData); - pub -> v_msg_pub(Msg, [MsgName], TrUserData); - async_call_reply -> v_msg_async_call_reply(Msg, [MsgName], TrUserData); - deploy -> v_msg_deploy(Msg, [MsgName], TrUserData); - fetch_task_log -> v_msg_fetch_task_log(Msg, [MsgName], TrUserData); - invoke -> v_msg_invoke(Msg, [MsgName], TrUserData); - push_service_config -> v_msg_push_service_config(Msg, [MsgName], TrUserData); - data -> v_msg_data(Msg, [MsgName], TrUserData); - ping -> v_msg_ping(Msg, [MsgName], TrUserData); - service_inform -> v_msg_service_inform(Msg, [MsgName], TrUserData); - event -> v_msg_event(Msg, [MsgName], TrUserData); - _ -> mk_type_error(not_a_known_message, Msg, []) - end. - - --compile({nowarn_unused_function,v_msg_auth_request/3}). --dialyzer({nowarn_function,v_msg_auth_request/3}). -v_msg_auth_request(#auth_request{uuid = F1, username = F2, salt = F3, token = F4, timestamp = F5}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [uuid | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [username | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_string(F3, [salt | Path], TrUserData) - end, - if F4 == undefined -> ok; - true -> v_type_string(F4, [token | Path], TrUserData) - end, - if F5 == undefined -> ok; - true -> v_type_uint32(F5, [timestamp | Path], TrUserData) - end, - ok; -v_msg_auth_request(X, Path, _TrUserData) -> mk_type_error({expected_msg, auth_request}, X, Path). - --compile({nowarn_unused_function,v_msg_auth_reply/3}). --dialyzer({nowarn_function,v_msg_auth_reply/3}). -v_msg_auth_reply(#auth_reply{code = F1, message = F2}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_uint32(F1, [code | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [message | Path], TrUserData) - end, - ok; -v_msg_auth_reply(X, Path, _TrUserData) -> mk_type_error({expected_msg, auth_reply}, X, Path). - --compile({nowarn_unused_function,v_msg_pub/3}). --dialyzer({nowarn_function,v_msg_pub/3}). -v_msg_pub(#pub{topic = F1, content = F2}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [topic | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [content | Path], TrUserData) - end, - ok; -v_msg_pub(X, Path, _TrUserData) -> mk_type_error({expected_msg, pub}, X, Path). - --compile({nowarn_unused_function,v_msg_async_call_reply/3}). --dialyzer({nowarn_function,v_msg_async_call_reply/3}). -v_msg_async_call_reply(#async_call_reply{code = F1, result = F2, message = F3}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_uint32(F1, [code | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [result | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_string(F3, [message | Path], TrUserData) - end, - ok; -v_msg_async_call_reply(X, Path, _TrUserData) -> mk_type_error({expected_msg, async_call_reply}, X, Path). - --compile({nowarn_unused_function,v_msg_deploy/3}). --dialyzer({nowarn_function,v_msg_deploy/3}). -v_msg_deploy(#deploy{task_id = F1, service_id = F2, tar_url = F3}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_uint32(F1, [task_id | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [service_id | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_string(F3, [tar_url | Path], TrUserData) - end, - ok; -v_msg_deploy(X, Path, _TrUserData) -> mk_type_error({expected_msg, deploy}, X, Path). - --compile({nowarn_unused_function,v_msg_fetch_task_log/3}). --dialyzer({nowarn_function,v_msg_fetch_task_log/3}). -v_msg_fetch_task_log(#fetch_task_log{task_id = F1}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_uint32(F1, [task_id | Path], TrUserData) - end, - ok; -v_msg_fetch_task_log(X, Path, _TrUserData) -> mk_type_error({expected_msg, fetch_task_log}, X, Path). - --compile({nowarn_unused_function,v_msg_invoke/3}). --dialyzer({nowarn_function,v_msg_invoke/3}). -v_msg_invoke(#invoke{service_id = F1, payload = F2, timeout = F3}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [service_id | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [payload | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_uint32(F3, [timeout | Path], TrUserData) - end, - ok; -v_msg_invoke(X, Path, _TrUserData) -> mk_type_error({expected_msg, invoke}, X, Path). - --compile({nowarn_unused_function,v_msg_push_service_config/3}). --dialyzer({nowarn_function,v_msg_push_service_config/3}). -v_msg_push_service_config(#push_service_config{service_id = F1, config_json = F2, timeout = F3}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [service_id | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [config_json | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_uint32(F3, [timeout | Path], TrUserData) - end, - ok; -v_msg_push_service_config(X, Path, _TrUserData) -> mk_type_error({expected_msg, push_service_config}, X, Path). - --compile({nowarn_unused_function,v_msg_data/3}). --dialyzer({nowarn_function,v_msg_data/3}). -v_msg_data(#data{service_id = F1, device_uuid = F2, metric = F3}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [service_id | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [device_uuid | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_string(F3, [metric | Path], TrUserData) - end, - ok; -v_msg_data(X, Path, _TrUserData) -> mk_type_error({expected_msg, data}, X, Path). - --compile({nowarn_unused_function,v_msg_ping/3}). --dialyzer({nowarn_function,v_msg_ping/3}). -v_msg_ping(#ping{adcode = F1, boot_time = F2, province = F3, city = F4, efka_version = F5, kernel_arch = F6, ips = F7, cpu_core = F8, cpu_load = F9, cpu_temperature = F10, disk = F11, memory = F12, interfaces = F13}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [adcode | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_uint32(F2, [boot_time | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_string(F3, [province | Path], TrUserData) - end, - if F4 == undefined -> ok; - true -> v_type_string(F4, [city | Path], TrUserData) - end, - if F5 == undefined -> ok; - true -> v_type_string(F5, [efka_version | Path], TrUserData) - end, - if F6 == undefined -> ok; - true -> v_type_string(F6, [kernel_arch | Path], TrUserData) - end, - if is_list(F7) -> - _ = [v_type_string(Elem, [ips | Path], TrUserData) || Elem <- F7], - ok; - true -> mk_type_error({invalid_list_of, string}, F7, [ips | Path]) - end, - if F8 == undefined -> ok; - true -> v_type_uint32(F8, [cpu_core | Path], TrUserData) - end, - if F9 == undefined -> ok; - true -> v_type_uint32(F9, [cpu_load | Path], TrUserData) - end, - if F10 == undefined -> ok; - true -> v_type_float(F10, [cpu_temperature | Path], TrUserData) - end, - if is_list(F11) -> - _ = [v_type_int32(Elem, [disk | Path], TrUserData) || Elem <- F11], - ok; - true -> mk_type_error({invalid_list_of, int32}, F11, [disk | Path]) - end, - if is_list(F12) -> - _ = [v_type_int32(Elem, [memory | Path], TrUserData) || Elem <- F12], - ok; - true -> mk_type_error({invalid_list_of, int32}, F12, [memory | Path]) - end, - if F13 == undefined -> ok; - true -> v_type_string(F13, [interfaces | Path], TrUserData) - end, - ok; -v_msg_ping(X, Path, _TrUserData) -> mk_type_error({expected_msg, ping}, X, Path). - --compile({nowarn_unused_function,v_msg_service_inform/3}). --dialyzer({nowarn_function,v_msg_service_inform/3}). -v_msg_service_inform(#service_inform{service_id = F1, props = F2, status = F3, timestamp = F4}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [service_id | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_string(F2, [props | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_uint32(F3, [status | Path], TrUserData) - end, - if F4 == undefined -> ok; - true -> v_type_uint32(F4, [timestamp | Path], TrUserData) - end, - ok; -v_msg_service_inform(X, Path, _TrUserData) -> mk_type_error({expected_msg, service_inform}, X, Path). - --compile({nowarn_unused_function,v_msg_event/3}). --dialyzer({nowarn_function,v_msg_event/3}). -v_msg_event(#event{service_id = F1, event_type = F2, params = F3}, Path, TrUserData) -> - if F1 == undefined -> ok; - true -> v_type_string(F1, [service_id | Path], TrUserData) - end, - if F2 == undefined -> ok; - true -> v_type_uint32(F2, [event_type | Path], TrUserData) - end, - if F3 == undefined -> ok; - true -> v_type_string(F3, [params | Path], TrUserData) - end, - ok; -v_msg_event(X, Path, _TrUserData) -> mk_type_error({expected_msg, event}, X, Path). - --compile({nowarn_unused_function,v_type_int32/3}). --dialyzer({nowarn_function,v_type_int32/3}). -v_type_int32(N, _Path, _TrUserData) when is_integer(N), -2147483648 =< N, N =< 2147483647 -> ok; -v_type_int32(N, Path, _TrUserData) when is_integer(N) -> mk_type_error({value_out_of_range, int32, signed, 32}, N, Path); -v_type_int32(X, Path, _TrUserData) -> mk_type_error({bad_integer, int32, signed, 32}, X, Path). - --compile({nowarn_unused_function,v_type_uint32/3}). --dialyzer({nowarn_function,v_type_uint32/3}). -v_type_uint32(N, _Path, _TrUserData) when is_integer(N), 0 =< N, N =< 4294967295 -> ok; -v_type_uint32(N, Path, _TrUserData) when is_integer(N) -> mk_type_error({value_out_of_range, uint32, unsigned, 32}, N, Path); -v_type_uint32(X, Path, _TrUserData) -> mk_type_error({bad_integer, uint32, unsigned, 32}, X, Path). - --compile({nowarn_unused_function,v_type_float/3}). --dialyzer({nowarn_function,v_type_float/3}). -v_type_float(N, _Path, _TrUserData) when is_float(N) -> ok; -v_type_float(N, _Path, _TrUserData) when is_integer(N) -> ok; -v_type_float(infinity, _Path, _TrUserData) -> ok; -v_type_float('-infinity', _Path, _TrUserData) -> ok; -v_type_float(nan, _Path, _TrUserData) -> ok; -v_type_float(X, Path, _TrUserData) -> mk_type_error(bad_float_value, X, Path). - --compile({nowarn_unused_function,v_type_string/3}). --dialyzer({nowarn_function,v_type_string/3}). -v_type_string(S, Path, _TrUserData) when is_list(S); is_binary(S) -> - try unicode:characters_to_binary(S) of - B when is_binary(B) -> ok; - {error, _, _} -> mk_type_error(bad_unicode_string, S, Path) - catch - error:badarg -> mk_type_error(bad_unicode_string, S, Path) - end; -v_type_string(X, Path, _TrUserData) -> mk_type_error(bad_unicode_string, X, Path). - --compile({nowarn_unused_function,mk_type_error/3}). --spec mk_type_error(_, _, list()) -> no_return(). -mk_type_error(Error, ValueSeen, Path) -> - Path2 = prettify_path(Path), - erlang:error({gpb_type_error, {Error, [{value, ValueSeen}, {path, Path2}]}}). - - --compile({nowarn_unused_function,prettify_path/1}). --dialyzer({nowarn_function,prettify_path/1}). -prettify_path([]) -> top_level; -prettify_path(PathR) -> lists:append(lists:join(".", lists:map(fun atom_to_list/1, lists:reverse(PathR)))). - - --compile({nowarn_unused_function,id/2}). --compile({inline,id/2}). -id(X, _TrUserData) -> X. - --compile({nowarn_unused_function,v_ok/3}). --compile({inline,v_ok/3}). -v_ok(_Value, _Path, _TrUserData) -> ok. - --compile({nowarn_unused_function,m_overwrite/3}). --compile({inline,m_overwrite/3}). -m_overwrite(_Prev, New, _TrUserData) -> New. - --compile({nowarn_unused_function,cons/3}). --compile({inline,cons/3}). -cons(Elem, Acc, _TrUserData) -> [Elem | Acc]. - --compile({nowarn_unused_function,lists_reverse/2}). --compile({inline,lists_reverse/2}). -'lists_reverse'(L, _TrUserData) -> lists:reverse(L). --compile({nowarn_unused_function,'erlang_++'/3}). --compile({inline,'erlang_++'/3}). -'erlang_++'(A, B, _TrUserData) -> A ++ B. - - -get_msg_defs() -> - [{{msg, auth_request}, - [#field{name = uuid, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = username, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = salt, fnum = 4, rnum = 4, type = string, occurrence = optional, opts = []}, - #field{name = token, fnum = 5, rnum = 5, type = string, occurrence = optional, opts = []}, - #field{name = timestamp, fnum = 6, rnum = 6, type = uint32, occurrence = optional, opts = []}]}, - {{msg, auth_reply}, [#field{name = code, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, #field{name = message, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}]}, - {{msg, pub}, [#field{name = topic, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, #field{name = content, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}]}, - {{msg, async_call_reply}, - [#field{name = code, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, - #field{name = result, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = message, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]}, - {{msg, deploy}, - [#field{name = task_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, - #field{name = service_id, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = tar_url, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]}, - {{msg, fetch_task_log}, [#field{name = task_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}]}, - {{msg, invoke}, - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = payload, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = timeout, fnum = 3, rnum = 4, type = uint32, occurrence = optional, opts = []}]}, - {{msg, push_service_config}, - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = config_json, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = timeout, fnum = 3, rnum = 4, type = uint32, occurrence = optional, opts = []}]}, - {{msg, data}, - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = device_uuid, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = metric, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]}, - {{msg, ping}, - [#field{name = adcode, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = boot_time, fnum = 2, rnum = 3, type = uint32, occurrence = optional, opts = []}, - #field{name = province, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}, - #field{name = city, fnum = 4, rnum = 5, type = string, occurrence = optional, opts = []}, - #field{name = efka_version, fnum = 5, rnum = 6, type = string, occurrence = optional, opts = []}, - #field{name = kernel_arch, fnum = 6, rnum = 7, type = string, occurrence = optional, opts = []}, - #field{name = ips, fnum = 7, rnum = 8, type = string, occurrence = repeated, opts = []}, - #field{name = cpu_core, fnum = 8, rnum = 9, type = uint32, occurrence = optional, opts = []}, - #field{name = cpu_load, fnum = 9, rnum = 10, type = uint32, occurrence = optional, opts = []}, - #field{name = cpu_temperature, fnum = 10, rnum = 11, type = float, occurrence = optional, opts = []}, - #field{name = disk, fnum = 11, rnum = 12, type = int32, occurrence = repeated, opts = [packed]}, - #field{name = memory, fnum = 12, rnum = 13, type = int32, occurrence = repeated, opts = [packed]}, - #field{name = interfaces, fnum = 13, rnum = 14, type = string, occurrence = optional, opts = []}]}, - {{msg, service_inform}, - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = props, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = status, fnum = 3, rnum = 4, type = uint32, occurrence = optional, opts = []}, - #field{name = timestamp, fnum = 4, rnum = 5, type = uint32, occurrence = optional, opts = []}]}, - {{msg, event}, - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = event_type, fnum = 2, rnum = 3, type = uint32, occurrence = optional, opts = []}, - #field{name = params, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]}]. - - -get_msg_names() -> [auth_request, auth_reply, pub, async_call_reply, deploy, fetch_task_log, invoke, push_service_config, data, ping, service_inform, event]. - - -get_group_names() -> []. - - -get_msg_or_group_names() -> [auth_request, auth_reply, pub, async_call_reply, deploy, fetch_task_log, invoke, push_service_config, data, ping, service_inform, event]. - - -get_enum_names() -> []. - - -fetch_msg_def(MsgName) -> - case find_msg_def(MsgName) of - Fs when is_list(Fs) -> Fs; - error -> erlang:error({no_such_msg, MsgName}) - end. - - --spec fetch_enum_def(_) -> no_return(). -fetch_enum_def(EnumName) -> erlang:error({no_such_enum, EnumName}). - - -find_msg_def(auth_request) -> - [#field{name = uuid, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = username, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = salt, fnum = 4, rnum = 4, type = string, occurrence = optional, opts = []}, - #field{name = token, fnum = 5, rnum = 5, type = string, occurrence = optional, opts = []}, - #field{name = timestamp, fnum = 6, rnum = 6, type = uint32, occurrence = optional, opts = []}]; -find_msg_def(auth_reply) -> [#field{name = code, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, #field{name = message, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}]; -find_msg_def(pub) -> [#field{name = topic, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, #field{name = content, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}]; -find_msg_def(async_call_reply) -> - [#field{name = code, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, - #field{name = result, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = message, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]; -find_msg_def(deploy) -> - [#field{name = task_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, - #field{name = service_id, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = tar_url, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]; -find_msg_def(fetch_task_log) -> [#field{name = task_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}]; -find_msg_def(invoke) -> - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = payload, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = timeout, fnum = 3, rnum = 4, type = uint32, occurrence = optional, opts = []}]; -find_msg_def(push_service_config) -> - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = config_json, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = timeout, fnum = 3, rnum = 4, type = uint32, occurrence = optional, opts = []}]; -find_msg_def(data) -> - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = device_uuid, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = metric, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]; -find_msg_def(ping) -> - [#field{name = adcode, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = boot_time, fnum = 2, rnum = 3, type = uint32, occurrence = optional, opts = []}, - #field{name = province, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}, - #field{name = city, fnum = 4, rnum = 5, type = string, occurrence = optional, opts = []}, - #field{name = efka_version, fnum = 5, rnum = 6, type = string, occurrence = optional, opts = []}, - #field{name = kernel_arch, fnum = 6, rnum = 7, type = string, occurrence = optional, opts = []}, - #field{name = ips, fnum = 7, rnum = 8, type = string, occurrence = repeated, opts = []}, - #field{name = cpu_core, fnum = 8, rnum = 9, type = uint32, occurrence = optional, opts = []}, - #field{name = cpu_load, fnum = 9, rnum = 10, type = uint32, occurrence = optional, opts = []}, - #field{name = cpu_temperature, fnum = 10, rnum = 11, type = float, occurrence = optional, opts = []}, - #field{name = disk, fnum = 11, rnum = 12, type = int32, occurrence = repeated, opts = [packed]}, - #field{name = memory, fnum = 12, rnum = 13, type = int32, occurrence = repeated, opts = [packed]}, - #field{name = interfaces, fnum = 13, rnum = 14, type = string, occurrence = optional, opts = []}]; -find_msg_def(service_inform) -> - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = props, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}, - #field{name = status, fnum = 3, rnum = 4, type = uint32, occurrence = optional, opts = []}, - #field{name = timestamp, fnum = 4, rnum = 5, type = uint32, occurrence = optional, opts = []}]; -find_msg_def(event) -> - [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, - #field{name = event_type, fnum = 2, rnum = 3, type = uint32, occurrence = optional, opts = []}, - #field{name = params, fnum = 3, rnum = 4, type = string, occurrence = optional, opts = []}]; -find_msg_def(_) -> error. - - -find_enum_def(_) -> error. - - --spec enum_symbol_by_value(_, _) -> no_return(). -enum_symbol_by_value(E, V) -> erlang:error({no_enum_defs, E, V}). - - --spec enum_value_by_symbol(_, _) -> no_return(). -enum_value_by_symbol(E, V) -> erlang:error({no_enum_defs, E, V}). - - - -get_service_names() -> []. - - -get_service_def(_) -> error. - - -get_rpc_names(_) -> error. - - -find_rpc_def(_, _) -> error. - - - --spec fetch_rpc_def(_, _) -> no_return(). -fetch_rpc_def(ServiceName, RpcName) -> erlang:error({no_such_rpc, ServiceName, RpcName}). - - -%% Convert a a fully qualified (ie with package name) service name -%% as a binary to a service name as an atom. --spec fqbin_to_service_name(_) -> no_return(). -fqbin_to_service_name(X) -> error({gpb_error, {badservice, X}}). - - -%% Convert a service name as an atom to a fully qualified -%% (ie with package name) name as a binary. --spec service_name_to_fqbin(_) -> no_return(). -service_name_to_fqbin(X) -> error({gpb_error, {badservice, X}}). - - -%% Convert a a fully qualified (ie with package name) service name -%% and an rpc name, both as binaries to a service name and an rpc -%% name, as atoms. --spec fqbins_to_service_and_rpc_name(_, _) -> no_return(). -fqbins_to_service_and_rpc_name(S, R) -> error({gpb_error, {badservice_or_rpc, {S, R}}}). - - -%% Convert a service name and an rpc name, both as atoms, -%% to a fully qualified (ie with package name) service name and -%% an rpc name as binaries. --spec service_and_rpc_name_to_fqbins(_, _) -> no_return(). -service_and_rpc_name_to_fqbins(S, R) -> error({gpb_error, {badservice_or_rpc, {S, R}}}). - - -fqbin_to_msg_name(<<"AuthRequest">>) -> auth_request; -fqbin_to_msg_name(<<"AuthReply">>) -> auth_reply; -fqbin_to_msg_name(<<"Pub">>) -> pub; -fqbin_to_msg_name(<<"AsyncCallReply">>) -> async_call_reply; -fqbin_to_msg_name(<<"Deploy">>) -> deploy; -fqbin_to_msg_name(<<"FetchTaskLog">>) -> fetch_task_log; -fqbin_to_msg_name(<<"Invoke">>) -> invoke; -fqbin_to_msg_name(<<"PushServiceConfig">>) -> push_service_config; -fqbin_to_msg_name(<<"Data">>) -> data; -fqbin_to_msg_name(<<"Ping">>) -> ping; -fqbin_to_msg_name(<<"ServiceInform">>) -> service_inform; -fqbin_to_msg_name(<<"Event">>) -> event; -fqbin_to_msg_name(E) -> error({gpb_error, {badmsg, E}}). - - -msg_name_to_fqbin(auth_request) -> <<"AuthRequest">>; -msg_name_to_fqbin(auth_reply) -> <<"AuthReply">>; -msg_name_to_fqbin(pub) -> <<"Pub">>; -msg_name_to_fqbin(async_call_reply) -> <<"AsyncCallReply">>; -msg_name_to_fqbin(deploy) -> <<"Deploy">>; -msg_name_to_fqbin(fetch_task_log) -> <<"FetchTaskLog">>; -msg_name_to_fqbin(invoke) -> <<"Invoke">>; -msg_name_to_fqbin(push_service_config) -> <<"PushServiceConfig">>; -msg_name_to_fqbin(data) -> <<"Data">>; -msg_name_to_fqbin(ping) -> <<"Ping">>; -msg_name_to_fqbin(service_inform) -> <<"ServiceInform">>; -msg_name_to_fqbin(event) -> <<"Event">>; -msg_name_to_fqbin(E) -> error({gpb_error, {badmsg, E}}). - - --spec fqbin_to_enum_name(_) -> no_return(). -fqbin_to_enum_name(E) -> error({gpb_error, {badenum, E}}). - - --spec enum_name_to_fqbin(_) -> no_return(). -enum_name_to_fqbin(E) -> error({gpb_error, {badenum, E}}). - - -get_package_name() -> undefined. - - -%% Whether or not the message names -%% are prepended with package name or not. -uses_packages() -> false. - - -source_basename() -> "message_pb.proto". - - -%% Retrieve all proto file names, also imported ones. -%% The order is top-down. The first element is always the main -%% source file. The files are returned with extension, -%% see get_all_proto_names/0 for a version that returns -%% the basenames sans extension -get_all_source_basenames() -> ["message_pb.proto"]. - - -%% Retrieve all proto file names, also imported ones. -%% The order is top-down. The first element is always the main -%% source file. The files are returned sans .proto extension, -%% to make it easier to use them with the various get_xyz_containment -%% functions. -get_all_proto_names() -> ["message_pb"]. - - -get_msg_containment("message_pb") -> [async_call_reply, auth_reply, auth_request, data, deploy, event, fetch_task_log, invoke, ping, pub, push_service_config, service_inform]; -get_msg_containment(P) -> error({gpb_error, {badproto, P}}). - - -get_pkg_containment("message_pb") -> undefined; -get_pkg_containment(P) -> error({gpb_error, {badproto, P}}). - - -get_service_containment("message_pb") -> []; -get_service_containment(P) -> error({gpb_error, {badproto, P}}). - - -get_rpc_containment("message_pb") -> []; -get_rpc_containment(P) -> error({gpb_error, {badproto, P}}). - - -get_enum_containment("message_pb") -> []; -get_enum_containment(P) -> error({gpb_error, {badproto, P}}). - - -get_proto_by_msg_name_as_fqbin(<<"Data">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"Pub">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"Event">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"AuthRequest">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"Invoke">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"PushServiceConfig">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"Ping">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"FetchTaskLog">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"Deploy">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"AuthReply">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"AsyncCallReply">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(<<"ServiceInform">>) -> "message_pb"; -get_proto_by_msg_name_as_fqbin(E) -> error({gpb_error, {badmsg, E}}). - - --spec get_proto_by_service_name_as_fqbin(_) -> no_return(). -get_proto_by_service_name_as_fqbin(E) -> error({gpb_error, {badservice, E}}). - - --spec get_proto_by_enum_name_as_fqbin(_) -> no_return(). -get_proto_by_enum_name_as_fqbin(E) -> error({gpb_error, {badenum, E}}). - - --spec get_protos_by_pkg_name_as_fqbin(_) -> no_return(). -get_protos_by_pkg_name_as_fqbin(E) -> error({gpb_error, {badpkg, E}}). - - - -gpb_version_as_string() -> - "4.21.1". - -gpb_version_as_list() -> - [4,21,1]. - -gpb_version_source() -> - "git". diff --git a/apps/efka/src/protobuf/efka_service_pb.erl b/apps/efka/src/protobuf/efka_service_pb.erl new file mode 100644 index 0000000..e08b796 --- /dev/null +++ b/apps/efka/src/protobuf/efka_service_pb.erl @@ -0,0 +1,1498 @@ +%% -*- coding: utf-8 -*- +%% @private +%% Automatically @generated, do not edit +%% Generated by gpb_compile version 4.21.7 +%% Version source: file +-module(efka_service_pb). + +-export([encode_msg/1, encode_msg/2, encode_msg/3]). +-export([decode_msg/2, decode_msg/3]). +-export([merge_msgs/2, merge_msgs/3, merge_msgs/4]). +-export([verify_msg/1, verify_msg/2, verify_msg/3]). +-export([get_msg_defs/0]). +-export([get_msg_names/0]). +-export([get_group_names/0]). +-export([get_msg_or_group_names/0]). +-export([get_enum_names/0]). +-export([find_msg_def/1, fetch_msg_def/1]). +-export([find_enum_def/1, fetch_enum_def/1]). +-export([enum_symbol_by_value/2, enum_value_by_symbol/2]). +-export([get_service_names/0]). +-export([get_service_def/1]). +-export([get_rpc_names/1]). +-export([find_rpc_def/2, fetch_rpc_def/2]). +-export([fqbin_to_service_name/1]). +-export([service_name_to_fqbin/1]). +-export([fqbins_to_service_and_rpc_name/2]). +-export([service_and_rpc_name_to_fqbins/2]). +-export([fqbin_to_msg_name/1]). +-export([msg_name_to_fqbin/1]). +-export([fqbin_to_enum_name/1]). +-export([enum_name_to_fqbin/1]). +-export([get_package_name/0]). +-export([uses_packages/0]). +-export([source_basename/0]). +-export([get_all_source_basenames/0]). +-export([get_all_proto_names/0]). +-export([get_msg_containment/1]). +-export([get_pkg_containment/1]). +-export([get_service_containment/1]). +-export([get_rpc_containment/1]). +-export([get_enum_containment/1]). +-export([get_proto_by_msg_name_as_fqbin/1]). +-export([get_proto_by_service_name_as_fqbin/1]). +-export([get_proto_by_enum_name_as_fqbin/1]). +-export([get_protos_by_pkg_name_as_fqbin/1]). +-export([gpb_version_as_string/0, gpb_version_as_list/0]). +-export([gpb_version_source/0]). + +-include("efka_service_pb.hrl"). +-include_lib("gpb/include/gpb.hrl"). + +%% enumerated types + +-export_type([]). + +%% message types +-type 'ServiceRequest.Register'() :: #'ServiceRequest.Register'{}. + +-type 'ServiceRequest.Subscribe'() :: #'ServiceRequest.Subscribe'{}. + +-type 'ServiceRequest'() :: #'ServiceRequest'{}. + +-type 'ServiceReply.Error'() :: #'ServiceReply.Error'{}. + +-type 'ServiceReply'() :: #'ServiceReply'{}. + +-type 'ServiceCast.MetricData'() :: #'ServiceCast.MetricData'{}. + +-type 'ServiceCast.TopicEvent'() :: #'ServiceCast.TopicEvent'{}. + +-type 'ServiceCast'() :: #'ServiceCast'{}. + +-export_type(['ServiceRequest.Register'/0, 'ServiceRequest.Subscribe'/0, 'ServiceRequest'/0, 'ServiceReply.Error'/0, 'ServiceReply'/0, 'ServiceCast.MetricData'/0, 'ServiceCast.TopicEvent'/0, 'ServiceCast'/0]). +-type '$msg_name'() :: 'ServiceRequest.Register' | 'ServiceRequest.Subscribe' | 'ServiceRequest' | 'ServiceReply.Error' | 'ServiceReply' | 'ServiceCast.MetricData' | 'ServiceCast.TopicEvent' | 'ServiceCast'. +-type '$msg'() :: 'ServiceRequest.Register'() | 'ServiceRequest.Subscribe'() | 'ServiceRequest'() | 'ServiceReply.Error'() | 'ServiceReply'() | 'ServiceCast.MetricData'() | 'ServiceCast.TopicEvent'() | 'ServiceCast'(). +-export_type(['$msg_name'/0, '$msg'/0]). + +-if(?OTP_RELEASE >= 24). +-dialyzer({no_underspecs, encode_msg/1}). +-endif. +-spec encode_msg('$msg'()) -> binary(). +encode_msg(Msg) when tuple_size(Msg) >= 1 -> encode_msg(Msg, element(1, Msg), []). + +-if(?OTP_RELEASE >= 24). +-dialyzer({no_underspecs, encode_msg/2}). +-endif. +-spec encode_msg('$msg'(), '$msg_name'() | list()) -> binary(). +encode_msg(Msg, MsgName) when is_atom(MsgName) -> encode_msg(Msg, MsgName, []); +encode_msg(Msg, Opts) when tuple_size(Msg) >= 1, is_list(Opts) -> encode_msg(Msg, element(1, Msg), Opts). + +-if(?OTP_RELEASE >= 24). +-dialyzer({no_underspecs, encode_msg/3}). +-endif. +-spec encode_msg('$msg'(), '$msg_name'(), list()) -> binary(). +encode_msg(Msg, MsgName, Opts) -> + case proplists:get_bool(verify, Opts) of + true -> verify_msg(Msg, MsgName, Opts); + false -> ok + end, + TrUserData = proplists:get_value(user_data, Opts), + case MsgName of + 'ServiceRequest.Register' -> 'encode_msg_ServiceRequest.Register'(id(Msg, TrUserData), TrUserData); + 'ServiceRequest.Subscribe' -> 'encode_msg_ServiceRequest.Subscribe'(id(Msg, TrUserData), TrUserData); + 'ServiceRequest' -> encode_msg_ServiceRequest(id(Msg, TrUserData), TrUserData); + 'ServiceReply.Error' -> 'encode_msg_ServiceReply.Error'(id(Msg, TrUserData), TrUserData); + 'ServiceReply' -> encode_msg_ServiceReply(id(Msg, TrUserData), TrUserData); + 'ServiceCast.MetricData' -> 'encode_msg_ServiceCast.MetricData'(id(Msg, TrUserData), TrUserData); + 'ServiceCast.TopicEvent' -> 'encode_msg_ServiceCast.TopicEvent'(id(Msg, TrUserData), TrUserData); + 'ServiceCast' -> encode_msg_ServiceCast(id(Msg, TrUserData), TrUserData) + end. + + +'encode_msg_ServiceRequest.Register'(Msg, TrUserData) -> 'encode_msg_ServiceRequest.Register'(Msg, <<>>, TrUserData). + + +'encode_msg_ServiceRequest.Register'(#'ServiceRequest.Register'{service_id = F1}, Bin, TrUserData) -> + if F1 == undefined -> Bin; + true -> + begin + TrF1 = id(F1, TrUserData), + case is_empty_string(TrF1) of + true -> Bin; + false -> e_type_string(TrF1, <>, TrUserData) + end + end + end. + +'encode_msg_ServiceRequest.Subscribe'(Msg, TrUserData) -> 'encode_msg_ServiceRequest.Subscribe'(Msg, <<>>, TrUserData). + + +'encode_msg_ServiceRequest.Subscribe'(#'ServiceRequest.Subscribe'{topic = F1}, Bin, TrUserData) -> + if F1 == undefined -> Bin; + true -> + begin + TrF1 = id(F1, TrUserData), + case is_empty_string(TrF1) of + true -> Bin; + false -> e_type_string(TrF1, <>, TrUserData) + end + end + end. + +encode_msg_ServiceRequest(Msg, TrUserData) -> encode_msg_ServiceRequest(Msg, <<>>, TrUserData). + + +encode_msg_ServiceRequest(#'ServiceRequest'{packet_id = F1, request = F2}, Bin, TrUserData) -> + B1 = if F1 == undefined -> Bin; + true -> + begin + TrF1 = id(F1, TrUserData), + if TrF1 =:= 0 -> Bin; + true -> e_varint(TrF1, <>, TrUserData) + end + end + end, + if F2 =:= undefined -> B1; + true -> + case id(F2, TrUserData) of + {register, TF2} -> begin TrTF2 = id(TF2, TrUserData), e_mfield_ServiceRequest_register(TrTF2, <>, TrUserData) end; + {subscribe, TF2} -> begin TrTF2 = id(TF2, TrUserData), e_mfield_ServiceRequest_subscribe(TrTF2, <>, TrUserData) end + end + end. + +'encode_msg_ServiceReply.Error'(Msg, TrUserData) -> 'encode_msg_ServiceReply.Error'(Msg, <<>>, TrUserData). + + +'encode_msg_ServiceReply.Error'(#'ServiceReply.Error'{code = F1, message = F2}, Bin, TrUserData) -> + B1 = if F1 == undefined -> Bin; + true -> + begin + TrF1 = id(F1, TrUserData), + if TrF1 =:= 0 -> Bin; + true -> e_type_int32(TrF1, <>, TrUserData) + end + end + end, + if F2 == undefined -> B1; + true -> + begin + TrF2 = id(F2, TrUserData), + case is_empty_string(TrF2) of + true -> B1; + false -> e_type_string(TrF2, <>, TrUserData) + end + end + end. + +encode_msg_ServiceReply(Msg, TrUserData) -> encode_msg_ServiceReply(Msg, <<>>, TrUserData). + + +encode_msg_ServiceReply(#'ServiceReply'{packet_id = F1, reply = F2}, Bin, TrUserData) -> + B1 = if F1 == undefined -> Bin; + true -> + begin + TrF1 = id(F1, TrUserData), + if TrF1 =:= 0 -> Bin; + true -> e_varint(TrF1, <>, TrUserData) + end + end + end, + if F2 =:= undefined -> B1; + true -> + case id(F2, TrUserData) of + {result, TF2} -> begin TrTF2 = id(TF2, TrUserData), e_type_bytes(TrTF2, <>, TrUserData) end; + {error, TF2} -> begin TrTF2 = id(TF2, TrUserData), e_mfield_ServiceReply_error(TrTF2, <>, TrUserData) end + end + end. + +'encode_msg_ServiceCast.MetricData'(Msg, TrUserData) -> 'encode_msg_ServiceCast.MetricData'(Msg, <<>>, TrUserData). + + +'encode_msg_ServiceCast.MetricData'(#'ServiceCast.MetricData'{route_key = F1, metric = F2}, Bin, TrUserData) -> + B1 = if F1 == undefined -> Bin; + true -> + begin + TrF1 = id(F1, TrUserData), + case iolist_size(TrF1) of + 0 -> Bin; + _ -> e_type_bytes(TrF1, <>, TrUserData) + end + end + end, + if F2 == undefined -> B1; + true -> + begin + TrF2 = id(F2, TrUserData), + case iolist_size(TrF2) of + 0 -> B1; + _ -> e_type_bytes(TrF2, <>, TrUserData) + end + end + end. + +'encode_msg_ServiceCast.TopicEvent'(Msg, TrUserData) -> 'encode_msg_ServiceCast.TopicEvent'(Msg, <<>>, TrUserData). + + +'encode_msg_ServiceCast.TopicEvent'(#'ServiceCast.TopicEvent'{topic = F1, content = F2}, Bin, TrUserData) -> + B1 = if F1 == undefined -> Bin; + true -> + begin + TrF1 = id(F1, TrUserData), + case is_empty_string(TrF1) of + true -> Bin; + false -> e_type_string(TrF1, <>, TrUserData) + end + end + end, + if F2 == undefined -> B1; + true -> + begin + TrF2 = id(F2, TrUserData), + case iolist_size(TrF2) of + 0 -> B1; + _ -> e_type_bytes(TrF2, <>, TrUserData) + end + end + end. + +encode_msg_ServiceCast(Msg, TrUserData) -> encode_msg_ServiceCast(Msg, <<>>, TrUserData). + + +encode_msg_ServiceCast(#'ServiceCast'{body = F1}, Bin, TrUserData) -> + if F1 =:= undefined -> Bin; + true -> + case id(F1, TrUserData) of + {topic_event, TF1} -> begin TrTF1 = id(TF1, TrUserData), e_mfield_ServiceCast_topic_event(TrTF1, <>, TrUserData) end; + {metric_data, TF1} -> begin TrTF1 = id(TF1, TrUserData), e_mfield_ServiceCast_metric_data(TrTF1, <>, TrUserData) end + end + end. + +e_mfield_ServiceRequest_register(Msg, Bin, TrUserData) -> + SubBin = 'encode_msg_ServiceRequest.Register'(Msg, <<>>, TrUserData), + Bin2 = e_varint(byte_size(SubBin), Bin), + <>. + +e_mfield_ServiceRequest_subscribe(Msg, Bin, TrUserData) -> + SubBin = 'encode_msg_ServiceRequest.Subscribe'(Msg, <<>>, TrUserData), + Bin2 = e_varint(byte_size(SubBin), Bin), + <>. + +e_mfield_ServiceReply_error(Msg, Bin, TrUserData) -> + SubBin = 'encode_msg_ServiceReply.Error'(Msg, <<>>, TrUserData), + Bin2 = e_varint(byte_size(SubBin), Bin), + <>. + +e_mfield_ServiceCast_topic_event(Msg, Bin, TrUserData) -> + SubBin = 'encode_msg_ServiceCast.TopicEvent'(Msg, <<>>, TrUserData), + Bin2 = e_varint(byte_size(SubBin), Bin), + <>. + +e_mfield_ServiceCast_metric_data(Msg, Bin, TrUserData) -> + SubBin = 'encode_msg_ServiceCast.MetricData'(Msg, <<>>, TrUserData), + Bin2 = e_varint(byte_size(SubBin), Bin), + <>. + +-compile({nowarn_unused_function,e_type_sint/3}). +e_type_sint(Value, Bin, _TrUserData) when Value >= 0 -> e_varint(Value * 2, Bin); +e_type_sint(Value, Bin, _TrUserData) -> e_varint(Value * -2 - 1, Bin). + +-compile({nowarn_unused_function,e_type_int32/3}). +e_type_int32(Value, Bin, _TrUserData) when 0 =< Value, Value =< 127 -> <>; +e_type_int32(Value, Bin, _TrUserData) -> + <> = <>, + e_varint(N, Bin). + +-compile({nowarn_unused_function,e_type_int64/3}). +e_type_int64(Value, Bin, _TrUserData) when 0 =< Value, Value =< 127 -> <>; +e_type_int64(Value, Bin, _TrUserData) -> + <> = <>, + e_varint(N, Bin). + +-compile({nowarn_unused_function,e_type_bool/3}). +e_type_bool(true, Bin, _TrUserData) -> <>; +e_type_bool(false, Bin, _TrUserData) -> <>; +e_type_bool(1, Bin, _TrUserData) -> <>; +e_type_bool(0, Bin, _TrUserData) -> <>. + +-compile({nowarn_unused_function,e_type_string/3}). +e_type_string(S, Bin, _TrUserData) -> + Utf8 = unicode:characters_to_binary(S), + Bin2 = e_varint(byte_size(Utf8), Bin), + <>. + +-compile({nowarn_unused_function,e_type_bytes/3}). +e_type_bytes(Bytes, Bin, _TrUserData) when is_binary(Bytes) -> + Bin2 = e_varint(byte_size(Bytes), Bin), + <>; +e_type_bytes(Bytes, Bin, _TrUserData) when is_list(Bytes) -> + BytesBin = iolist_to_binary(Bytes), + Bin2 = e_varint(byte_size(BytesBin), Bin), + <>. + +-compile({nowarn_unused_function,e_type_fixed32/3}). +e_type_fixed32(Value, Bin, _TrUserData) -> <>. + +-compile({nowarn_unused_function,e_type_sfixed32/3}). +e_type_sfixed32(Value, Bin, _TrUserData) -> <>. + +-compile({nowarn_unused_function,e_type_fixed64/3}). +e_type_fixed64(Value, Bin, _TrUserData) -> <>. + +-compile({nowarn_unused_function,e_type_sfixed64/3}). +e_type_sfixed64(Value, Bin, _TrUserData) -> <>. + +-compile({nowarn_unused_function,e_type_float/3}). +e_type_float(V, Bin, _) when is_number(V) -> <>; +e_type_float(infinity, Bin, _) -> <>; +e_type_float('-infinity', Bin, _) -> <>; +e_type_float(nan, Bin, _) -> <>. + +-compile({nowarn_unused_function,e_type_double/3}). +e_type_double(V, Bin, _) when is_number(V) -> <>; +e_type_double(infinity, Bin, _) -> <>; +e_type_double('-infinity', Bin, _) -> <>; +e_type_double(nan, Bin, _) -> <>. + +-compile({nowarn_unused_function,e_unknown_elems/2}). +e_unknown_elems([Elem | Rest], Bin) -> + BinR = case Elem of + {varint, FNum, N} -> + BinF = e_varint(FNum bsl 3, Bin), + e_varint(N, BinF); + {length_delimited, FNum, Data} -> + BinF = e_varint(FNum bsl 3 bor 2, Bin), + BinL = e_varint(byte_size(Data), BinF), + <>; + {group, FNum, GroupFields} -> + Bin1 = e_varint(FNum bsl 3 bor 3, Bin), + Bin2 = e_unknown_elems(GroupFields, Bin1), + e_varint(FNum bsl 3 bor 4, Bin2); + {fixed32, FNum, V} -> + BinF = e_varint(FNum bsl 3 bor 5, Bin), + <>; + {fixed64, FNum, V} -> + BinF = e_varint(FNum bsl 3 bor 1, Bin), + <> + end, + e_unknown_elems(Rest, BinR); +e_unknown_elems([], Bin) -> Bin. + +-compile({nowarn_unused_function,e_varint/3}). +e_varint(N, Bin, _TrUserData) -> e_varint(N, Bin). + +-compile({nowarn_unused_function,e_varint/2}). +e_varint(N, Bin) when N =< 127 -> <>; +e_varint(N, Bin) -> + Bin2 = <>, + e_varint(N bsr 7, Bin2). + +is_empty_string("") -> true; +is_empty_string(<<>>) -> true; +is_empty_string(L) when is_list(L) -> not string_has_chars(L); +is_empty_string(B) when is_binary(B) -> false. + +string_has_chars([C | _]) when is_integer(C) -> true; +string_has_chars([H | T]) -> + case string_has_chars(H) of + true -> true; + false -> string_has_chars(T) + end; +string_has_chars(B) when is_binary(B), byte_size(B) =/= 0 -> true; +string_has_chars(C) when is_integer(C) -> true; +string_has_chars(<<>>) -> false; +string_has_chars([]) -> false. + + +decode_msg(Bin, MsgName) when is_binary(Bin) -> decode_msg(Bin, MsgName, []). + +decode_msg(Bin, MsgName, Opts) when is_binary(Bin) -> + TrUserData = proplists:get_value(user_data, Opts), + decode_msg_1_catch(Bin, MsgName, TrUserData). + +-ifdef('OTP_RELEASE'). +decode_msg_1_catch(Bin, MsgName, TrUserData) -> + try decode_msg_2_doit(MsgName, Bin, TrUserData) + catch + error:{gpb_error,_}=Reason:StackTrace -> + erlang:raise(error, Reason, StackTrace); + Class:Reason:StackTrace -> error({gpb_error,{decoding_failure, {Bin, MsgName, {Class, Reason, StackTrace}}}}) + end. +-else. +decode_msg_1_catch(Bin, MsgName, TrUserData) -> + try decode_msg_2_doit(MsgName, Bin, TrUserData) + catch + error:{gpb_error,_}=Reason -> + erlang:raise(error, Reason, + erlang:get_stacktrace()); + Class:Reason -> + StackTrace = erlang:get_stacktrace(), + error({gpb_error,{decoding_failure, {Bin, MsgName, {Class, Reason, StackTrace}}}}) + end. +-endif. + +decode_msg_2_doit('ServiceRequest.Register', Bin, TrUserData) -> id('decode_msg_ServiceRequest.Register'(Bin, TrUserData), TrUserData); +decode_msg_2_doit('ServiceRequest.Subscribe', Bin, TrUserData) -> id('decode_msg_ServiceRequest.Subscribe'(Bin, TrUserData), TrUserData); +decode_msg_2_doit('ServiceRequest', Bin, TrUserData) -> id(decode_msg_ServiceRequest(Bin, TrUserData), TrUserData); +decode_msg_2_doit('ServiceReply.Error', Bin, TrUserData) -> id('decode_msg_ServiceReply.Error'(Bin, TrUserData), TrUserData); +decode_msg_2_doit('ServiceReply', Bin, TrUserData) -> id(decode_msg_ServiceReply(Bin, TrUserData), TrUserData); +decode_msg_2_doit('ServiceCast.MetricData', Bin, TrUserData) -> id('decode_msg_ServiceCast.MetricData'(Bin, TrUserData), TrUserData); +decode_msg_2_doit('ServiceCast.TopicEvent', Bin, TrUserData) -> id('decode_msg_ServiceCast.TopicEvent'(Bin, TrUserData), TrUserData); +decode_msg_2_doit('ServiceCast', Bin, TrUserData) -> id(decode_msg_ServiceCast(Bin, TrUserData), TrUserData). + + + +'decode_msg_ServiceRequest.Register'(Bin, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Register'(Bin, 0, 0, 0, id(<<>>, TrUserData), TrUserData). + +'dfp_read_field_def_ServiceRequest.Register'(<<10, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'd_field_ServiceRequest.Register_service_id'(Rest, Z1, Z2, F, F@_1, TrUserData); +'dfp_read_field_def_ServiceRequest.Register'(<<>>, 0, 0, _, F@_1, _) -> #'ServiceRequest.Register'{service_id = F@_1}; +'dfp_read_field_def_ServiceRequest.Register'(Other, Z1, Z2, F, F@_1, TrUserData) -> 'dg_read_field_def_ServiceRequest.Register'(Other, Z1, Z2, F, F@_1, TrUserData). + +'dg_read_field_def_ServiceRequest.Register'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 32 - 7 -> 'dg_read_field_def_ServiceRequest.Register'(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +'dg_read_field_def_ServiceRequest.Register'(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 10 -> 'd_field_ServiceRequest.Register_service_id'(Rest, 0, 0, 0, F@_1, TrUserData); + _ -> + case Key band 7 of + 0 -> 'skip_varint_ServiceRequest.Register'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 1 -> 'skip_64_ServiceRequest.Register'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 2 -> 'skip_length_delimited_ServiceRequest.Register'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 3 -> 'skip_group_ServiceRequest.Register'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 5 -> 'skip_32_ServiceRequest.Register'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData) + end + end; +'dg_read_field_def_ServiceRequest.Register'(<<>>, 0, 0, _, F@_1, _) -> #'ServiceRequest.Register'{service_id = F@_1}. + +'d_field_ServiceRequest.Register_service_id'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> 'd_field_ServiceRequest.Register_service_id'(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +'d_field_ServiceRequest.Register_service_id'(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + 'dfp_read_field_def_ServiceRequest.Register'(RestF, 0, 0, F, NewFValue, TrUserData). + +'skip_varint_ServiceRequest.Register'(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'skip_varint_ServiceRequest.Register'(Rest, Z1, Z2, F, F@_1, TrUserData); +'skip_varint_ServiceRequest.Register'(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Register'(Rest, Z1, Z2, F, F@_1, TrUserData). + +'skip_length_delimited_ServiceRequest.Register'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> 'skip_length_delimited_ServiceRequest.Register'(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +'skip_length_delimited_ServiceRequest.Register'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + 'dfp_read_field_def_ServiceRequest.Register'(Rest2, 0, 0, F, F@_1, TrUserData). + +'skip_group_ServiceRequest.Register'(Bin, _, Z2, FNum, F@_1, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + 'dfp_read_field_def_ServiceRequest.Register'(Rest, 0, Z2, FNum, F@_1, TrUserData). + +'skip_32_ServiceRequest.Register'(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Register'(Rest, Z1, Z2, F, F@_1, TrUserData). + +'skip_64_ServiceRequest.Register'(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Register'(Rest, Z1, Z2, F, F@_1, TrUserData). + +'decode_msg_ServiceRequest.Subscribe'(Bin, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Subscribe'(Bin, 0, 0, 0, id(<<>>, TrUserData), TrUserData). + +'dfp_read_field_def_ServiceRequest.Subscribe'(<<10, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'd_field_ServiceRequest.Subscribe_topic'(Rest, Z1, Z2, F, F@_1, TrUserData); +'dfp_read_field_def_ServiceRequest.Subscribe'(<<>>, 0, 0, _, F@_1, _) -> #'ServiceRequest.Subscribe'{topic = F@_1}; +'dfp_read_field_def_ServiceRequest.Subscribe'(Other, Z1, Z2, F, F@_1, TrUserData) -> 'dg_read_field_def_ServiceRequest.Subscribe'(Other, Z1, Z2, F, F@_1, TrUserData). + +'dg_read_field_def_ServiceRequest.Subscribe'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 32 - 7 -> 'dg_read_field_def_ServiceRequest.Subscribe'(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +'dg_read_field_def_ServiceRequest.Subscribe'(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 10 -> 'd_field_ServiceRequest.Subscribe_topic'(Rest, 0, 0, 0, F@_1, TrUserData); + _ -> + case Key band 7 of + 0 -> 'skip_varint_ServiceRequest.Subscribe'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 1 -> 'skip_64_ServiceRequest.Subscribe'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 2 -> 'skip_length_delimited_ServiceRequest.Subscribe'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 3 -> 'skip_group_ServiceRequest.Subscribe'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 5 -> 'skip_32_ServiceRequest.Subscribe'(Rest, 0, 0, Key bsr 3, F@_1, TrUserData) + end + end; +'dg_read_field_def_ServiceRequest.Subscribe'(<<>>, 0, 0, _, F@_1, _) -> #'ServiceRequest.Subscribe'{topic = F@_1}. + +'d_field_ServiceRequest.Subscribe_topic'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> 'd_field_ServiceRequest.Subscribe_topic'(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +'d_field_ServiceRequest.Subscribe_topic'(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + 'dfp_read_field_def_ServiceRequest.Subscribe'(RestF, 0, 0, F, NewFValue, TrUserData). + +'skip_varint_ServiceRequest.Subscribe'(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'skip_varint_ServiceRequest.Subscribe'(Rest, Z1, Z2, F, F@_1, TrUserData); +'skip_varint_ServiceRequest.Subscribe'(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Subscribe'(Rest, Z1, Z2, F, F@_1, TrUserData). + +'skip_length_delimited_ServiceRequest.Subscribe'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> 'skip_length_delimited_ServiceRequest.Subscribe'(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +'skip_length_delimited_ServiceRequest.Subscribe'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + 'dfp_read_field_def_ServiceRequest.Subscribe'(Rest2, 0, 0, F, F@_1, TrUserData). + +'skip_group_ServiceRequest.Subscribe'(Bin, _, Z2, FNum, F@_1, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + 'dfp_read_field_def_ServiceRequest.Subscribe'(Rest, 0, Z2, FNum, F@_1, TrUserData). + +'skip_32_ServiceRequest.Subscribe'(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Subscribe'(Rest, Z1, Z2, F, F@_1, TrUserData). + +'skip_64_ServiceRequest.Subscribe'(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> 'dfp_read_field_def_ServiceRequest.Subscribe'(Rest, Z1, Z2, F, F@_1, TrUserData). + +decode_msg_ServiceRequest(Bin, TrUserData) -> dfp_read_field_def_ServiceRequest(Bin, 0, 0, 0, id(0, TrUserData), id(undefined, TrUserData), TrUserData). + +dfp_read_field_def_ServiceRequest(<<8, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_ServiceRequest_packet_id(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +dfp_read_field_def_ServiceRequest(<<82, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_ServiceRequest_register(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +dfp_read_field_def_ServiceRequest(<<90, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_ServiceRequest_subscribe(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +dfp_read_field_def_ServiceRequest(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceRequest'{packet_id = F@_1, request = F@_2}; +dfp_read_field_def_ServiceRequest(Other, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dg_read_field_def_ServiceRequest(Other, Z1, Z2, F, F@_1, F@_2, TrUserData). + +dg_read_field_def_ServiceRequest(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 32 - 7 -> dg_read_field_def_ServiceRequest(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +dg_read_field_def_ServiceRequest(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 8 -> d_field_ServiceRequest_packet_id(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + 82 -> d_field_ServiceRequest_register(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + 90 -> d_field_ServiceRequest_subscribe(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + _ -> + case Key band 7 of + 0 -> skip_varint_ServiceRequest(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 1 -> skip_64_ServiceRequest(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 2 -> skip_length_delimited_ServiceRequest(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 3 -> skip_group_ServiceRequest(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 5 -> skip_32_ServiceRequest(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData) + end + end; +dg_read_field_def_ServiceRequest(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceRequest'{packet_id = F@_1, request = F@_2}. + +d_field_ServiceRequest_packet_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_ServiceRequest_packet_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +d_field_ServiceRequest_packet_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, TrUserData) -> + {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, + dfp_read_field_def_ServiceRequest(RestF, 0, 0, F, NewFValue, F@_2, TrUserData). + +d_field_ServiceRequest_register(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_ServiceRequest_register(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +d_field_ServiceRequest_register(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, Prev, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, {id('decode_msg_ServiceRequest.Register'(Bs, TrUserData), TrUserData), Rest2} end, + dfp_read_field_def_ServiceRequest(RestF, + 0, + 0, + F, + F@_1, + case Prev of + undefined -> id({register, NewFValue}, TrUserData); + {register, MVPrev} -> id({register, 'merge_msg_ServiceRequest.Register'(MVPrev, NewFValue, TrUserData)}, TrUserData); + _ -> id({register, NewFValue}, TrUserData) + end, + TrUserData). + +d_field_ServiceRequest_subscribe(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_ServiceRequest_subscribe(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +d_field_ServiceRequest_subscribe(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, Prev, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, {id('decode_msg_ServiceRequest.Subscribe'(Bs, TrUserData), TrUserData), Rest2} end, + dfp_read_field_def_ServiceRequest(RestF, + 0, + 0, + F, + F@_1, + case Prev of + undefined -> id({subscribe, NewFValue}, TrUserData); + {subscribe, MVPrev} -> id({subscribe, 'merge_msg_ServiceRequest.Subscribe'(MVPrev, NewFValue, TrUserData)}, TrUserData); + _ -> id({subscribe, NewFValue}, TrUserData) + end, + TrUserData). + +skip_varint_ServiceRequest(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> skip_varint_ServiceRequest(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +skip_varint_ServiceRequest(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_ServiceRequest(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +skip_length_delimited_ServiceRequest(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> skip_length_delimited_ServiceRequest(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +skip_length_delimited_ServiceRequest(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + dfp_read_field_def_ServiceRequest(Rest2, 0, 0, F, F@_1, F@_2, TrUserData). + +skip_group_ServiceRequest(Bin, _, Z2, FNum, F@_1, F@_2, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + dfp_read_field_def_ServiceRequest(Rest, 0, Z2, FNum, F@_1, F@_2, TrUserData). + +skip_32_ServiceRequest(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_ServiceRequest(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +skip_64_ServiceRequest(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_ServiceRequest(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'decode_msg_ServiceReply.Error'(Bin, TrUserData) -> 'dfp_read_field_def_ServiceReply.Error'(Bin, 0, 0, 0, id(0, TrUserData), id(<<>>, TrUserData), TrUserData). + +'dfp_read_field_def_ServiceReply.Error'(<<8, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'd_field_ServiceReply.Error_code'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'dfp_read_field_def_ServiceReply.Error'(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'd_field_ServiceReply.Error_message'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'dfp_read_field_def_ServiceReply.Error'(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceReply.Error'{code = F@_1, message = F@_2}; +'dfp_read_field_def_ServiceReply.Error'(Other, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dg_read_field_def_ServiceReply.Error'(Other, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'dg_read_field_def_ServiceReply.Error'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 32 - 7 -> 'dg_read_field_def_ServiceReply.Error'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'dg_read_field_def_ServiceReply.Error'(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 8 -> 'd_field_ServiceReply.Error_code'(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + 18 -> 'd_field_ServiceReply.Error_message'(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + _ -> + case Key band 7 of + 0 -> 'skip_varint_ServiceReply.Error'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 1 -> 'skip_64_ServiceReply.Error'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 2 -> 'skip_length_delimited_ServiceReply.Error'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 3 -> 'skip_group_ServiceReply.Error'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 5 -> 'skip_32_ServiceReply.Error'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData) + end + end; +'dg_read_field_def_ServiceReply.Error'(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceReply.Error'{code = F@_1, message = F@_2}. + +'d_field_ServiceReply.Error_code'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'd_field_ServiceReply.Error_code'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'d_field_ServiceReply.Error_code'(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, TrUserData) -> + {NewFValue, RestF} = {begin <> = <<(X bsl N + Acc):32/unsigned-native>>, id(Res, TrUserData) end, Rest}, + 'dfp_read_field_def_ServiceReply.Error'(RestF, 0, 0, F, NewFValue, F@_2, TrUserData). + +'d_field_ServiceReply.Error_message'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'd_field_ServiceReply.Error_message'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'d_field_ServiceReply.Error_message'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + 'dfp_read_field_def_ServiceReply.Error'(RestF, 0, 0, F, F@_1, NewFValue, TrUserData). + +'skip_varint_ServiceReply.Error'(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'skip_varint_ServiceReply.Error'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'skip_varint_ServiceReply.Error'(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceReply.Error'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'skip_length_delimited_ServiceReply.Error'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'skip_length_delimited_ServiceReply.Error'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'skip_length_delimited_ServiceReply.Error'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + 'dfp_read_field_def_ServiceReply.Error'(Rest2, 0, 0, F, F@_1, F@_2, TrUserData). + +'skip_group_ServiceReply.Error'(Bin, _, Z2, FNum, F@_1, F@_2, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + 'dfp_read_field_def_ServiceReply.Error'(Rest, 0, Z2, FNum, F@_1, F@_2, TrUserData). + +'skip_32_ServiceReply.Error'(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceReply.Error'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'skip_64_ServiceReply.Error'(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceReply.Error'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +decode_msg_ServiceReply(Bin, TrUserData) -> dfp_read_field_def_ServiceReply(Bin, 0, 0, 0, id(0, TrUserData), id(undefined, TrUserData), TrUserData). + +dfp_read_field_def_ServiceReply(<<8, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_ServiceReply_packet_id(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +dfp_read_field_def_ServiceReply(<<82, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_ServiceReply_result(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +dfp_read_field_def_ServiceReply(<<90, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> d_field_ServiceReply_error(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +dfp_read_field_def_ServiceReply(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceReply'{packet_id = F@_1, reply = F@_2}; +dfp_read_field_def_ServiceReply(Other, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dg_read_field_def_ServiceReply(Other, Z1, Z2, F, F@_1, F@_2, TrUserData). + +dg_read_field_def_ServiceReply(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 32 - 7 -> dg_read_field_def_ServiceReply(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +dg_read_field_def_ServiceReply(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 8 -> d_field_ServiceReply_packet_id(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + 82 -> d_field_ServiceReply_result(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + 90 -> d_field_ServiceReply_error(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + _ -> + case Key band 7 of + 0 -> skip_varint_ServiceReply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 1 -> skip_64_ServiceReply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 2 -> skip_length_delimited_ServiceReply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 3 -> skip_group_ServiceReply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 5 -> skip_32_ServiceReply(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData) + end + end; +dg_read_field_def_ServiceReply(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceReply'{packet_id = F@_1, reply = F@_2}. + +d_field_ServiceReply_packet_id(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_ServiceReply_packet_id(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +d_field_ServiceReply_packet_id(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, TrUserData) -> + {NewFValue, RestF} = {id((X bsl N + Acc) band 4294967295, TrUserData), Rest}, + dfp_read_field_def_ServiceReply(RestF, 0, 0, F, NewFValue, F@_2, TrUserData). + +d_field_ServiceReply_result(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_ServiceReply_result(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +d_field_ServiceReply_result(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + dfp_read_field_def_ServiceReply(RestF, 0, 0, F, F@_1, id({result, NewFValue}, TrUserData), TrUserData). + +d_field_ServiceReply_error(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> d_field_ServiceReply_error(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +d_field_ServiceReply_error(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, Prev, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, {id('decode_msg_ServiceReply.Error'(Bs, TrUserData), TrUserData), Rest2} end, + dfp_read_field_def_ServiceReply(RestF, + 0, + 0, + F, + F@_1, + case Prev of + undefined -> id({error, NewFValue}, TrUserData); + {error, MVPrev} -> id({error, 'merge_msg_ServiceReply.Error'(MVPrev, NewFValue, TrUserData)}, TrUserData); + _ -> id({error, NewFValue}, TrUserData) + end, + TrUserData). + +skip_varint_ServiceReply(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> skip_varint_ServiceReply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +skip_varint_ServiceReply(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_ServiceReply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +skip_length_delimited_ServiceReply(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> skip_length_delimited_ServiceReply(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +skip_length_delimited_ServiceReply(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + dfp_read_field_def_ServiceReply(Rest2, 0, 0, F, F@_1, F@_2, TrUserData). + +skip_group_ServiceReply(Bin, _, Z2, FNum, F@_1, F@_2, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + dfp_read_field_def_ServiceReply(Rest, 0, Z2, FNum, F@_1, F@_2, TrUserData). + +skip_32_ServiceReply(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_ServiceReply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +skip_64_ServiceReply(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> dfp_read_field_def_ServiceReply(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'decode_msg_ServiceCast.MetricData'(Bin, TrUserData) -> 'dfp_read_field_def_ServiceCast.MetricData'(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), TrUserData). + +'dfp_read_field_def_ServiceCast.MetricData'(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'd_field_ServiceCast.MetricData_route_key'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'dfp_read_field_def_ServiceCast.MetricData'(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'd_field_ServiceCast.MetricData_metric'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'dfp_read_field_def_ServiceCast.MetricData'(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceCast.MetricData'{route_key = F@_1, metric = F@_2}; +'dfp_read_field_def_ServiceCast.MetricData'(Other, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dg_read_field_def_ServiceCast.MetricData'(Other, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'dg_read_field_def_ServiceCast.MetricData'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 32 - 7 -> 'dg_read_field_def_ServiceCast.MetricData'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'dg_read_field_def_ServiceCast.MetricData'(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 10 -> 'd_field_ServiceCast.MetricData_route_key'(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + 18 -> 'd_field_ServiceCast.MetricData_metric'(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + _ -> + case Key band 7 of + 0 -> 'skip_varint_ServiceCast.MetricData'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 1 -> 'skip_64_ServiceCast.MetricData'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 2 -> 'skip_length_delimited_ServiceCast.MetricData'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 3 -> 'skip_group_ServiceCast.MetricData'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 5 -> 'skip_32_ServiceCast.MetricData'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData) + end + end; +'dg_read_field_def_ServiceCast.MetricData'(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceCast.MetricData'{route_key = F@_1, metric = F@_2}. + +'d_field_ServiceCast.MetricData_route_key'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'd_field_ServiceCast.MetricData_route_key'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'d_field_ServiceCast.MetricData_route_key'(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + 'dfp_read_field_def_ServiceCast.MetricData'(RestF, 0, 0, F, NewFValue, F@_2, TrUserData). + +'d_field_ServiceCast.MetricData_metric'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'd_field_ServiceCast.MetricData_metric'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'d_field_ServiceCast.MetricData_metric'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + 'dfp_read_field_def_ServiceCast.MetricData'(RestF, 0, 0, F, F@_1, NewFValue, TrUserData). + +'skip_varint_ServiceCast.MetricData'(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'skip_varint_ServiceCast.MetricData'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'skip_varint_ServiceCast.MetricData'(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceCast.MetricData'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'skip_length_delimited_ServiceCast.MetricData'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'skip_length_delimited_ServiceCast.MetricData'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'skip_length_delimited_ServiceCast.MetricData'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + 'dfp_read_field_def_ServiceCast.MetricData'(Rest2, 0, 0, F, F@_1, F@_2, TrUserData). + +'skip_group_ServiceCast.MetricData'(Bin, _, Z2, FNum, F@_1, F@_2, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + 'dfp_read_field_def_ServiceCast.MetricData'(Rest, 0, Z2, FNum, F@_1, F@_2, TrUserData). + +'skip_32_ServiceCast.MetricData'(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceCast.MetricData'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'skip_64_ServiceCast.MetricData'(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceCast.MetricData'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'decode_msg_ServiceCast.TopicEvent'(Bin, TrUserData) -> 'dfp_read_field_def_ServiceCast.TopicEvent'(Bin, 0, 0, 0, id(<<>>, TrUserData), id(<<>>, TrUserData), TrUserData). + +'dfp_read_field_def_ServiceCast.TopicEvent'(<<10, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'd_field_ServiceCast.TopicEvent_topic'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'dfp_read_field_def_ServiceCast.TopicEvent'(<<18, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'd_field_ServiceCast.TopicEvent_content'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'dfp_read_field_def_ServiceCast.TopicEvent'(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceCast.TopicEvent'{topic = F@_1, content = F@_2}; +'dfp_read_field_def_ServiceCast.TopicEvent'(Other, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dg_read_field_def_ServiceCast.TopicEvent'(Other, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'dg_read_field_def_ServiceCast.TopicEvent'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 32 - 7 -> 'dg_read_field_def_ServiceCast.TopicEvent'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'dg_read_field_def_ServiceCast.TopicEvent'(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, F@_2, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 10 -> 'd_field_ServiceCast.TopicEvent_topic'(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + 18 -> 'd_field_ServiceCast.TopicEvent_content'(Rest, 0, 0, 0, F@_1, F@_2, TrUserData); + _ -> + case Key band 7 of + 0 -> 'skip_varint_ServiceCast.TopicEvent'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 1 -> 'skip_64_ServiceCast.TopicEvent'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 2 -> 'skip_length_delimited_ServiceCast.TopicEvent'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 3 -> 'skip_group_ServiceCast.TopicEvent'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData); + 5 -> 'skip_32_ServiceCast.TopicEvent'(Rest, 0, 0, Key bsr 3, F@_1, F@_2, TrUserData) + end + end; +'dg_read_field_def_ServiceCast.TopicEvent'(<<>>, 0, 0, _, F@_1, F@_2, _) -> #'ServiceCast.TopicEvent'{topic = F@_1, content = F@_2}. + +'d_field_ServiceCast.TopicEvent_topic'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'd_field_ServiceCast.TopicEvent_topic'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'d_field_ServiceCast.TopicEvent_topic'(<<0:1, X:7, Rest/binary>>, N, Acc, F, _, F@_2, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + 'dfp_read_field_def_ServiceCast.TopicEvent'(RestF, 0, 0, F, NewFValue, F@_2, TrUserData). + +'d_field_ServiceCast.TopicEvent_content'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'd_field_ServiceCast.TopicEvent_content'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'d_field_ServiceCast.TopicEvent_content'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, _, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, Bytes2 = binary:copy(Bytes), {id(Bytes2, TrUserData), Rest2} end, + 'dfp_read_field_def_ServiceCast.TopicEvent'(RestF, 0, 0, F, F@_1, NewFValue, TrUserData). + +'skip_varint_ServiceCast.TopicEvent'(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'skip_varint_ServiceCast.TopicEvent'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData); +'skip_varint_ServiceCast.TopicEvent'(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceCast.TopicEvent'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'skip_length_delimited_ServiceCast.TopicEvent'(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) when N < 57 -> 'skip_length_delimited_ServiceCast.TopicEvent'(Rest, N + 7, X bsl N + Acc, F, F@_1, F@_2, TrUserData); +'skip_length_delimited_ServiceCast.TopicEvent'(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, F@_2, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + 'dfp_read_field_def_ServiceCast.TopicEvent'(Rest2, 0, 0, F, F@_1, F@_2, TrUserData). + +'skip_group_ServiceCast.TopicEvent'(Bin, _, Z2, FNum, F@_1, F@_2, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + 'dfp_read_field_def_ServiceCast.TopicEvent'(Rest, 0, Z2, FNum, F@_1, F@_2, TrUserData). + +'skip_32_ServiceCast.TopicEvent'(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceCast.TopicEvent'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +'skip_64_ServiceCast.TopicEvent'(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, F@_2, TrUserData) -> 'dfp_read_field_def_ServiceCast.TopicEvent'(Rest, Z1, Z2, F, F@_1, F@_2, TrUserData). + +decode_msg_ServiceCast(Bin, TrUserData) -> dfp_read_field_def_ServiceCast(Bin, 0, 0, 0, id(undefined, TrUserData), TrUserData). + +dfp_read_field_def_ServiceCast(<<82, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> d_field_ServiceCast_topic_event(Rest, Z1, Z2, F, F@_1, TrUserData); +dfp_read_field_def_ServiceCast(<<90, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> d_field_ServiceCast_metric_data(Rest, Z1, Z2, F, F@_1, TrUserData); +dfp_read_field_def_ServiceCast(<<>>, 0, 0, _, F@_1, _) -> #'ServiceCast'{body = F@_1}; +dfp_read_field_def_ServiceCast(Other, Z1, Z2, F, F@_1, TrUserData) -> dg_read_field_def_ServiceCast(Other, Z1, Z2, F, F@_1, TrUserData). + +dg_read_field_def_ServiceCast(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 32 - 7 -> dg_read_field_def_ServiceCast(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +dg_read_field_def_ServiceCast(<<0:1, X:7, Rest/binary>>, N, Acc, _, F@_1, TrUserData) -> + Key = X bsl N + Acc, + case Key of + 82 -> d_field_ServiceCast_topic_event(Rest, 0, 0, 0, F@_1, TrUserData); + 90 -> d_field_ServiceCast_metric_data(Rest, 0, 0, 0, F@_1, TrUserData); + _ -> + case Key band 7 of + 0 -> skip_varint_ServiceCast(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 1 -> skip_64_ServiceCast(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 2 -> skip_length_delimited_ServiceCast(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 3 -> skip_group_ServiceCast(Rest, 0, 0, Key bsr 3, F@_1, TrUserData); + 5 -> skip_32_ServiceCast(Rest, 0, 0, Key bsr 3, F@_1, TrUserData) + end + end; +dg_read_field_def_ServiceCast(<<>>, 0, 0, _, F@_1, _) -> #'ServiceCast'{body = F@_1}. + +d_field_ServiceCast_topic_event(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> d_field_ServiceCast_topic_event(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +d_field_ServiceCast_topic_event(<<0:1, X:7, Rest/binary>>, N, Acc, F, Prev, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, {id('decode_msg_ServiceCast.TopicEvent'(Bs, TrUserData), TrUserData), Rest2} end, + dfp_read_field_def_ServiceCast(RestF, + 0, + 0, + F, + case Prev of + undefined -> id({topic_event, NewFValue}, TrUserData); + {topic_event, MVPrev} -> id({topic_event, 'merge_msg_ServiceCast.TopicEvent'(MVPrev, NewFValue, TrUserData)}, TrUserData); + _ -> id({topic_event, NewFValue}, TrUserData) + end, + TrUserData). + +d_field_ServiceCast_metric_data(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> d_field_ServiceCast_metric_data(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +d_field_ServiceCast_metric_data(<<0:1, X:7, Rest/binary>>, N, Acc, F, Prev, TrUserData) -> + {NewFValue, RestF} = begin Len = X bsl N + Acc, <> = Rest, {id('decode_msg_ServiceCast.MetricData'(Bs, TrUserData), TrUserData), Rest2} end, + dfp_read_field_def_ServiceCast(RestF, + 0, + 0, + F, + case Prev of + undefined -> id({metric_data, NewFValue}, TrUserData); + {metric_data, MVPrev} -> id({metric_data, 'merge_msg_ServiceCast.MetricData'(MVPrev, NewFValue, TrUserData)}, TrUserData); + _ -> id({metric_data, NewFValue}, TrUserData) + end, + TrUserData). + +skip_varint_ServiceCast(<<1:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> skip_varint_ServiceCast(Rest, Z1, Z2, F, F@_1, TrUserData); +skip_varint_ServiceCast(<<0:1, _:7, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> dfp_read_field_def_ServiceCast(Rest, Z1, Z2, F, F@_1, TrUserData). + +skip_length_delimited_ServiceCast(<<1:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) when N < 57 -> skip_length_delimited_ServiceCast(Rest, N + 7, X bsl N + Acc, F, F@_1, TrUserData); +skip_length_delimited_ServiceCast(<<0:1, X:7, Rest/binary>>, N, Acc, F, F@_1, TrUserData) -> + Length = X bsl N + Acc, + <<_:Length/binary, Rest2/binary>> = Rest, + dfp_read_field_def_ServiceCast(Rest2, 0, 0, F, F@_1, TrUserData). + +skip_group_ServiceCast(Bin, _, Z2, FNum, F@_1, TrUserData) -> + {_, Rest} = read_group(Bin, FNum), + dfp_read_field_def_ServiceCast(Rest, 0, Z2, FNum, F@_1, TrUserData). + +skip_32_ServiceCast(<<_:32, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> dfp_read_field_def_ServiceCast(Rest, Z1, Z2, F, F@_1, TrUserData). + +skip_64_ServiceCast(<<_:64, Rest/binary>>, Z1, Z2, F, F@_1, TrUserData) -> dfp_read_field_def_ServiceCast(Rest, Z1, Z2, F, F@_1, TrUserData). + +read_group(Bin, FieldNum) -> + {NumBytes, EndTagLen} = read_gr_b(Bin, 0, 0, 0, 0, FieldNum), + <> = Bin, + {Group, Rest}. + +%% Like skipping over fields, but record the total length, +%% Each field is <(FieldNum bsl 3) bor FieldType> ++ +%% Record the length because varints may be non-optimally encoded. +%% +%% Groups can be nested, but assume the same FieldNum cannot be nested +%% because group field numbers are shared with the rest of the fields +%% numbers. Thus we can search just for an group-end with the same +%% field number. +%% +%% (The only time the same group field number could occur would +%% be in a nested sub message, but then it would be inside a +%% length-delimited entry, which we skip-read by length.) +read_gr_b(<<1:1, X:7, Tl/binary>>, N, Acc, NumBytes, TagLen, FieldNum) + when N < (32-7) -> + read_gr_b(Tl, N+7, X bsl N + Acc, NumBytes, TagLen+1, FieldNum); +read_gr_b(<<0:1, X:7, Tl/binary>>, N, Acc, NumBytes, TagLen, + FieldNum) -> + Key = X bsl N + Acc, + TagLen1 = TagLen + 1, + case {Key bsr 3, Key band 7} of + {FieldNum, 4} -> % 4 = group_end + {NumBytes, TagLen1}; + {_, 0} -> % 0 = varint + read_gr_vi(Tl, 0, NumBytes + TagLen1, FieldNum); + {_, 1} -> % 1 = bits64 + <<_:64, Tl2/binary>> = Tl, + read_gr_b(Tl2, 0, 0, NumBytes + TagLen1 + 8, 0, FieldNum); + {_, 2} -> % 2 = length_delimited + read_gr_ld(Tl, 0, 0, NumBytes + TagLen1, FieldNum); + {_, 3} -> % 3 = group_start + read_gr_b(Tl, 0, 0, NumBytes + TagLen1, 0, FieldNum); + {_, 4} -> % 4 = group_end + read_gr_b(Tl, 0, 0, NumBytes + TagLen1, 0, FieldNum); + {_, 5} -> % 5 = bits32 + <<_:32, Tl2/binary>> = Tl, + read_gr_b(Tl2, 0, 0, NumBytes + TagLen1 + 4, 0, FieldNum) + end. + +read_gr_vi(<<1:1, _:7, Tl/binary>>, N, NumBytes, FieldNum) + when N < (64-7) -> + read_gr_vi(Tl, N+7, NumBytes+1, FieldNum); +read_gr_vi(<<0:1, _:7, Tl/binary>>, _, NumBytes, FieldNum) -> + read_gr_b(Tl, 0, 0, NumBytes+1, 0, FieldNum). + +read_gr_ld(<<1:1, X:7, Tl/binary>>, N, Acc, NumBytes, FieldNum) + when N < (64-7) -> + read_gr_ld(Tl, N+7, X bsl N + Acc, NumBytes+1, FieldNum); +read_gr_ld(<<0:1, X:7, Tl/binary>>, N, Acc, NumBytes, FieldNum) -> + Len = X bsl N + Acc, + NumBytes1 = NumBytes + 1, + <<_:Len/binary, Tl2/binary>> = Tl, + read_gr_b(Tl2, 0, 0, NumBytes1 + Len, 0, FieldNum). + +merge_msgs(Prev, New) when element(1, Prev) =:= element(1, New) -> merge_msgs(Prev, New, element(1, Prev), []). + +merge_msgs(Prev, New, MsgName) when is_atom(MsgName) -> merge_msgs(Prev, New, MsgName, []); +merge_msgs(Prev, New, Opts) when element(1, Prev) =:= element(1, New), is_list(Opts) -> merge_msgs(Prev, New, element(1, Prev), Opts). + +merge_msgs(Prev, New, MsgName, Opts) -> + TrUserData = proplists:get_value(user_data, Opts), + case MsgName of + 'ServiceRequest.Register' -> 'merge_msg_ServiceRequest.Register'(Prev, New, TrUserData); + 'ServiceRequest.Subscribe' -> 'merge_msg_ServiceRequest.Subscribe'(Prev, New, TrUserData); + 'ServiceRequest' -> merge_msg_ServiceRequest(Prev, New, TrUserData); + 'ServiceReply.Error' -> 'merge_msg_ServiceReply.Error'(Prev, New, TrUserData); + 'ServiceReply' -> merge_msg_ServiceReply(Prev, New, TrUserData); + 'ServiceCast.MetricData' -> 'merge_msg_ServiceCast.MetricData'(Prev, New, TrUserData); + 'ServiceCast.TopicEvent' -> 'merge_msg_ServiceCast.TopicEvent'(Prev, New, TrUserData); + 'ServiceCast' -> merge_msg_ServiceCast(Prev, New, TrUserData) + end. + +-compile({nowarn_unused_function,'merge_msg_ServiceRequest.Register'/3}). +'merge_msg_ServiceRequest.Register'(#'ServiceRequest.Register'{service_id = PFservice_id}, #'ServiceRequest.Register'{service_id = NFservice_id}, _) -> + #'ServiceRequest.Register'{service_id = + if NFservice_id =:= undefined -> PFservice_id; + true -> NFservice_id + end}. + +-compile({nowarn_unused_function,'merge_msg_ServiceRequest.Subscribe'/3}). +'merge_msg_ServiceRequest.Subscribe'(#'ServiceRequest.Subscribe'{topic = PFtopic}, #'ServiceRequest.Subscribe'{topic = NFtopic}, _) -> + #'ServiceRequest.Subscribe'{topic = + if NFtopic =:= undefined -> PFtopic; + true -> NFtopic + end}. + +-compile({nowarn_unused_function,merge_msg_ServiceRequest/3}). +merge_msg_ServiceRequest(#'ServiceRequest'{packet_id = PFpacket_id, request = PFrequest}, #'ServiceRequest'{packet_id = NFpacket_id, request = NFrequest}, TrUserData) -> + #'ServiceRequest'{packet_id = + if NFpacket_id =:= undefined -> PFpacket_id; + true -> NFpacket_id + end, + request = + case {PFrequest, NFrequest} of + {{register, OPFrequest}, {register, ONFrequest}} -> {register, 'merge_msg_ServiceRequest.Register'(OPFrequest, ONFrequest, TrUserData)}; + {{subscribe, OPFrequest}, {subscribe, ONFrequest}} -> {subscribe, 'merge_msg_ServiceRequest.Subscribe'(OPFrequest, ONFrequest, TrUserData)}; + {_, undefined} -> PFrequest; + _ -> NFrequest + end}. + +-compile({nowarn_unused_function,'merge_msg_ServiceReply.Error'/3}). +'merge_msg_ServiceReply.Error'(#'ServiceReply.Error'{code = PFcode, message = PFmessage}, #'ServiceReply.Error'{code = NFcode, message = NFmessage}, _) -> + #'ServiceReply.Error'{code = + if NFcode =:= undefined -> PFcode; + true -> NFcode + end, + message = + if NFmessage =:= undefined -> PFmessage; + true -> NFmessage + end}. + +-compile({nowarn_unused_function,merge_msg_ServiceReply/3}). +merge_msg_ServiceReply(#'ServiceReply'{packet_id = PFpacket_id, reply = PFreply}, #'ServiceReply'{packet_id = NFpacket_id, reply = NFreply}, TrUserData) -> + #'ServiceReply'{packet_id = + if NFpacket_id =:= undefined -> PFpacket_id; + true -> NFpacket_id + end, + reply = + case {PFreply, NFreply} of + {{error, OPFreply}, {error, ONFreply}} -> {error, 'merge_msg_ServiceReply.Error'(OPFreply, ONFreply, TrUserData)}; + {_, undefined} -> PFreply; + _ -> NFreply + end}. + +-compile({nowarn_unused_function,'merge_msg_ServiceCast.MetricData'/3}). +'merge_msg_ServiceCast.MetricData'(#'ServiceCast.MetricData'{route_key = PFroute_key, metric = PFmetric}, #'ServiceCast.MetricData'{route_key = NFroute_key, metric = NFmetric}, _) -> + #'ServiceCast.MetricData'{route_key = + if NFroute_key =:= undefined -> PFroute_key; + true -> NFroute_key + end, + metric = + if NFmetric =:= undefined -> PFmetric; + true -> NFmetric + end}. + +-compile({nowarn_unused_function,'merge_msg_ServiceCast.TopicEvent'/3}). +'merge_msg_ServiceCast.TopicEvent'(#'ServiceCast.TopicEvent'{topic = PFtopic, content = PFcontent}, #'ServiceCast.TopicEvent'{topic = NFtopic, content = NFcontent}, _) -> + #'ServiceCast.TopicEvent'{topic = + if NFtopic =:= undefined -> PFtopic; + true -> NFtopic + end, + content = + if NFcontent =:= undefined -> PFcontent; + true -> NFcontent + end}. + +-compile({nowarn_unused_function,merge_msg_ServiceCast/3}). +merge_msg_ServiceCast(#'ServiceCast'{body = PFbody}, #'ServiceCast'{body = NFbody}, TrUserData) -> + #'ServiceCast'{body = + case {PFbody, NFbody} of + {{topic_event, OPFbody}, {topic_event, ONFbody}} -> {topic_event, 'merge_msg_ServiceCast.TopicEvent'(OPFbody, ONFbody, TrUserData)}; + {{metric_data, OPFbody}, {metric_data, ONFbody}} -> {metric_data, 'merge_msg_ServiceCast.MetricData'(OPFbody, ONFbody, TrUserData)}; + {_, undefined} -> PFbody; + _ -> NFbody + end}. + + +verify_msg(Msg) when tuple_size(Msg) >= 1 -> verify_msg(Msg, element(1, Msg), []); +verify_msg(X) -> mk_type_error(not_a_known_message, X, []). + +verify_msg(Msg, MsgName) when is_atom(MsgName) -> verify_msg(Msg, MsgName, []); +verify_msg(Msg, Opts) when tuple_size(Msg) >= 1 -> verify_msg(Msg, element(1, Msg), Opts); +verify_msg(X, _Opts) -> mk_type_error(not_a_known_message, X, []). + +verify_msg(Msg, MsgName, Opts) -> + TrUserData = proplists:get_value(user_data, Opts), + case MsgName of + 'ServiceRequest.Register' -> 'v_msg_ServiceRequest.Register'(Msg, [MsgName], TrUserData); + 'ServiceRequest.Subscribe' -> 'v_msg_ServiceRequest.Subscribe'(Msg, [MsgName], TrUserData); + 'ServiceRequest' -> v_msg_ServiceRequest(Msg, [MsgName], TrUserData); + 'ServiceReply.Error' -> 'v_msg_ServiceReply.Error'(Msg, [MsgName], TrUserData); + 'ServiceReply' -> v_msg_ServiceReply(Msg, [MsgName], TrUserData); + 'ServiceCast.MetricData' -> 'v_msg_ServiceCast.MetricData'(Msg, [MsgName], TrUserData); + 'ServiceCast.TopicEvent' -> 'v_msg_ServiceCast.TopicEvent'(Msg, [MsgName], TrUserData); + 'ServiceCast' -> v_msg_ServiceCast(Msg, [MsgName], TrUserData); + _ -> mk_type_error(not_a_known_message, Msg, []) + end. + + +-compile({nowarn_unused_function,'v_submsg_ServiceRequest.Register'/3}). +-dialyzer({nowarn_function,'v_submsg_ServiceRequest.Register'/3}). +'v_submsg_ServiceRequest.Register'(Msg, Path, TrUserData) -> 'v_msg_ServiceRequest.Register'(Msg, Path, TrUserData). + +-compile({nowarn_unused_function,'v_msg_ServiceRequest.Register'/3}). +-dialyzer({nowarn_function,'v_msg_ServiceRequest.Register'/3}). +'v_msg_ServiceRequest.Register'(#'ServiceRequest.Register'{service_id = F1}, Path, TrUserData) -> + if F1 == undefined -> ok; + true -> v_type_string(F1, [service_id | Path], TrUserData) + end, + ok; +'v_msg_ServiceRequest.Register'(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceRequest.Register'}, X, Path). + +-compile({nowarn_unused_function,'v_submsg_ServiceRequest.Subscribe'/3}). +-dialyzer({nowarn_function,'v_submsg_ServiceRequest.Subscribe'/3}). +'v_submsg_ServiceRequest.Subscribe'(Msg, Path, TrUserData) -> 'v_msg_ServiceRequest.Subscribe'(Msg, Path, TrUserData). + +-compile({nowarn_unused_function,'v_msg_ServiceRequest.Subscribe'/3}). +-dialyzer({nowarn_function,'v_msg_ServiceRequest.Subscribe'/3}). +'v_msg_ServiceRequest.Subscribe'(#'ServiceRequest.Subscribe'{topic = F1}, Path, TrUserData) -> + if F1 == undefined -> ok; + true -> v_type_string(F1, [topic | Path], TrUserData) + end, + ok; +'v_msg_ServiceRequest.Subscribe'(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceRequest.Subscribe'}, X, Path). + +-compile({nowarn_unused_function,v_msg_ServiceRequest/3}). +-dialyzer({nowarn_function,v_msg_ServiceRequest/3}). +v_msg_ServiceRequest(#'ServiceRequest'{packet_id = F1, request = F2}, Path, TrUserData) -> + if F1 == undefined -> ok; + true -> v_type_uint32(F1, [packet_id | Path], TrUserData) + end, + case F2 of + undefined -> ok; + {register, OF2} -> 'v_submsg_ServiceRequest.Register'(OF2, [register, request | Path], TrUserData); + {subscribe, OF2} -> 'v_submsg_ServiceRequest.Subscribe'(OF2, [subscribe, request | Path], TrUserData); + _ -> mk_type_error(invalid_oneof, F2, [request | Path]) + end, + ok; +v_msg_ServiceRequest(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceRequest'}, X, Path). + +-compile({nowarn_unused_function,'v_submsg_ServiceReply.Error'/3}). +-dialyzer({nowarn_function,'v_submsg_ServiceReply.Error'/3}). +'v_submsg_ServiceReply.Error'(Msg, Path, TrUserData) -> 'v_msg_ServiceReply.Error'(Msg, Path, TrUserData). + +-compile({nowarn_unused_function,'v_msg_ServiceReply.Error'/3}). +-dialyzer({nowarn_function,'v_msg_ServiceReply.Error'/3}). +'v_msg_ServiceReply.Error'(#'ServiceReply.Error'{code = F1, message = F2}, Path, TrUserData) -> + if F1 == undefined -> ok; + true -> v_type_int32(F1, [code | Path], TrUserData) + end, + if F2 == undefined -> ok; + true -> v_type_string(F2, [message | Path], TrUserData) + end, + ok; +'v_msg_ServiceReply.Error'(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceReply.Error'}, X, Path). + +-compile({nowarn_unused_function,v_msg_ServiceReply/3}). +-dialyzer({nowarn_function,v_msg_ServiceReply/3}). +v_msg_ServiceReply(#'ServiceReply'{packet_id = F1, reply = F2}, Path, TrUserData) -> + if F1 == undefined -> ok; + true -> v_type_uint32(F1, [packet_id | Path], TrUserData) + end, + case F2 of + undefined -> ok; + {result, OF2} -> v_type_bytes(OF2, [result, reply | Path], TrUserData); + {error, OF2} -> 'v_submsg_ServiceReply.Error'(OF2, [error, reply | Path], TrUserData); + _ -> mk_type_error(invalid_oneof, F2, [reply | Path]) + end, + ok; +v_msg_ServiceReply(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceReply'}, X, Path). + +-compile({nowarn_unused_function,'v_submsg_ServiceCast.MetricData'/3}). +-dialyzer({nowarn_function,'v_submsg_ServiceCast.MetricData'/3}). +'v_submsg_ServiceCast.MetricData'(Msg, Path, TrUserData) -> 'v_msg_ServiceCast.MetricData'(Msg, Path, TrUserData). + +-compile({nowarn_unused_function,'v_msg_ServiceCast.MetricData'/3}). +-dialyzer({nowarn_function,'v_msg_ServiceCast.MetricData'/3}). +'v_msg_ServiceCast.MetricData'(#'ServiceCast.MetricData'{route_key = F1, metric = F2}, Path, TrUserData) -> + if F1 == undefined -> ok; + true -> v_type_bytes(F1, [route_key | Path], TrUserData) + end, + if F2 == undefined -> ok; + true -> v_type_bytes(F2, [metric | Path], TrUserData) + end, + ok; +'v_msg_ServiceCast.MetricData'(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceCast.MetricData'}, X, Path). + +-compile({nowarn_unused_function,'v_submsg_ServiceCast.TopicEvent'/3}). +-dialyzer({nowarn_function,'v_submsg_ServiceCast.TopicEvent'/3}). +'v_submsg_ServiceCast.TopicEvent'(Msg, Path, TrUserData) -> 'v_msg_ServiceCast.TopicEvent'(Msg, Path, TrUserData). + +-compile({nowarn_unused_function,'v_msg_ServiceCast.TopicEvent'/3}). +-dialyzer({nowarn_function,'v_msg_ServiceCast.TopicEvent'/3}). +'v_msg_ServiceCast.TopicEvent'(#'ServiceCast.TopicEvent'{topic = F1, content = F2}, Path, TrUserData) -> + if F1 == undefined -> ok; + true -> v_type_string(F1, [topic | Path], TrUserData) + end, + if F2 == undefined -> ok; + true -> v_type_bytes(F2, [content | Path], TrUserData) + end, + ok; +'v_msg_ServiceCast.TopicEvent'(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceCast.TopicEvent'}, X, Path). + +-compile({nowarn_unused_function,v_msg_ServiceCast/3}). +-dialyzer({nowarn_function,v_msg_ServiceCast/3}). +v_msg_ServiceCast(#'ServiceCast'{body = F1}, Path, TrUserData) -> + case F1 of + undefined -> ok; + {topic_event, OF1} -> 'v_submsg_ServiceCast.TopicEvent'(OF1, [topic_event, body | Path], TrUserData); + {metric_data, OF1} -> 'v_submsg_ServiceCast.MetricData'(OF1, [metric_data, body | Path], TrUserData); + _ -> mk_type_error(invalid_oneof, F1, [body | Path]) + end, + ok; +v_msg_ServiceCast(X, Path, _TrUserData) -> mk_type_error({expected_msg, 'ServiceCast'}, X, Path). + +-compile({nowarn_unused_function,v_type_int32/3}). +-dialyzer({nowarn_function,v_type_int32/3}). +v_type_int32(N, _Path, _TrUserData) when is_integer(N), -2147483648 =< N, N =< 2147483647 -> ok; +v_type_int32(N, Path, _TrUserData) when is_integer(N) -> mk_type_error({value_out_of_range, int32, signed, 32}, N, Path); +v_type_int32(X, Path, _TrUserData) -> mk_type_error({bad_integer, int32, signed, 32}, X, Path). + +-compile({nowarn_unused_function,v_type_uint32/3}). +-dialyzer({nowarn_function,v_type_uint32/3}). +v_type_uint32(N, _Path, _TrUserData) when is_integer(N), 0 =< N, N =< 4294967295 -> ok; +v_type_uint32(N, Path, _TrUserData) when is_integer(N) -> mk_type_error({value_out_of_range, uint32, unsigned, 32}, N, Path); +v_type_uint32(X, Path, _TrUserData) -> mk_type_error({bad_integer, uint32, unsigned, 32}, X, Path). + +-compile({nowarn_unused_function,v_type_string/3}). +-dialyzer({nowarn_function,v_type_string/3}). +v_type_string(S, Path, _TrUserData) when is_list(S); is_binary(S) -> + try unicode:characters_to_binary(S) of + B when is_binary(B) -> ok; + {error, _, _} -> mk_type_error(bad_unicode_string, S, Path) + catch + error:badarg -> mk_type_error(bad_unicode_string, S, Path) + end; +v_type_string(X, Path, _TrUserData) -> mk_type_error(bad_unicode_string, X, Path). + +-compile({nowarn_unused_function,v_type_bytes/3}). +-dialyzer({nowarn_function,v_type_bytes/3}). +v_type_bytes(B, _Path, _TrUserData) when is_binary(B) -> ok; +v_type_bytes(B, _Path, _TrUserData) when is_list(B) -> ok; +v_type_bytes(X, Path, _TrUserData) -> mk_type_error(bad_binary_value, X, Path). + +-compile({nowarn_unused_function,mk_type_error/3}). +-spec mk_type_error(_, _, list()) -> no_return(). +mk_type_error(Error, ValueSeen, Path) -> + Path2 = prettify_path(Path), + erlang:error({gpb_type_error, {Error, [{value, ValueSeen}, {path, Path2}]}}). + + +-compile({nowarn_unused_function,prettify_path/1}). +-dialyzer({nowarn_function,prettify_path/1}). +prettify_path([]) -> top_level; +prettify_path(PathR) -> lists:append(lists:join(".", lists:map(fun atom_to_list/1, lists:reverse(PathR)))). + + +-compile({nowarn_unused_function,id/2}). +-compile({inline,id/2}). +id(X, _TrUserData) -> X. + +-compile({nowarn_unused_function,v_ok/3}). +-compile({inline,v_ok/3}). +v_ok(_Value, _Path, _TrUserData) -> ok. + +-compile({nowarn_unused_function,m_overwrite/3}). +-compile({inline,m_overwrite/3}). +m_overwrite(_Prev, New, _TrUserData) -> New. + +-compile({nowarn_unused_function,cons/3}). +-compile({inline,cons/3}). +cons(Elem, Acc, _TrUserData) -> [Elem | Acc]. + +-compile({nowarn_unused_function,lists_reverse/2}). +-compile({inline,lists_reverse/2}). +'lists_reverse'(L, _TrUserData) -> lists:reverse(L). +-compile({nowarn_unused_function,'erlang_++'/3}). +-compile({inline,'erlang_++'/3}). +'erlang_++'(A, B, _TrUserData) -> A ++ B. + + +get_msg_defs() -> + [{{msg, 'ServiceRequest.Register'}, [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}]}, + {{msg, 'ServiceRequest.Subscribe'}, [#field{name = topic, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}]}, + {{msg, 'ServiceRequest'}, + [#field{name = packet_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, + #gpb_oneof{name = request, rnum = 3, + fields = + [#field{name = register, fnum = 10, rnum = 3, type = {msg, 'ServiceRequest.Register'}, occurrence = optional, opts = []}, #field{name = subscribe, fnum = 11, rnum = 3, type = {msg, 'ServiceRequest.Subscribe'}, occurrence = optional, opts = []}], + opts = []}]}, + {{msg, 'ServiceReply.Error'}, [#field{name = code, fnum = 1, rnum = 2, type = int32, occurrence = optional, opts = []}, #field{name = message, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}]}, + {{msg, 'ServiceReply'}, + [#field{name = packet_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, + #gpb_oneof{name = reply, rnum = 3, fields = [#field{name = result, fnum = 10, rnum = 3, type = bytes, occurrence = optional, opts = []}, #field{name = error, fnum = 11, rnum = 3, type = {msg, 'ServiceReply.Error'}, occurrence = optional, opts = []}], + opts = []}]}, + {{msg, 'ServiceCast.MetricData'}, [#field{name = route_key, fnum = 1, rnum = 2, type = bytes, occurrence = optional, opts = []}, #field{name = metric, fnum = 2, rnum = 3, type = bytes, occurrence = optional, opts = []}]}, + {{msg, 'ServiceCast.TopicEvent'}, [#field{name = topic, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, #field{name = content, fnum = 2, rnum = 3, type = bytes, occurrence = optional, opts = []}]}, + {{msg, 'ServiceCast'}, + [#gpb_oneof{name = body, rnum = 2, + fields = + [#field{name = topic_event, fnum = 10, rnum = 2, type = {msg, 'ServiceCast.TopicEvent'}, occurrence = optional, opts = []}, #field{name = metric_data, fnum = 11, rnum = 2, type = {msg, 'ServiceCast.MetricData'}, occurrence = optional, opts = []}], + opts = []}]}]. + + +get_msg_names() -> ['ServiceRequest.Register', 'ServiceRequest.Subscribe', 'ServiceRequest', 'ServiceReply.Error', 'ServiceReply', 'ServiceCast.MetricData', 'ServiceCast.TopicEvent', 'ServiceCast']. + + +get_group_names() -> []. + + +get_msg_or_group_names() -> ['ServiceRequest.Register', 'ServiceRequest.Subscribe', 'ServiceRequest', 'ServiceReply.Error', 'ServiceReply', 'ServiceCast.MetricData', 'ServiceCast.TopicEvent', 'ServiceCast']. + + +get_enum_names() -> []. + + +fetch_msg_def(MsgName) -> + case find_msg_def(MsgName) of + Fs when is_list(Fs) -> Fs; + error -> erlang:error({no_such_msg, MsgName}) + end. + + +-spec fetch_enum_def(_) -> no_return(). +fetch_enum_def(EnumName) -> erlang:error({no_such_enum, EnumName}). + + +find_msg_def('ServiceRequest.Register') -> [#field{name = service_id, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}]; +find_msg_def('ServiceRequest.Subscribe') -> [#field{name = topic, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}]; +find_msg_def('ServiceRequest') -> + [#field{name = packet_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, + #gpb_oneof{name = request, rnum = 3, + fields = + [#field{name = register, fnum = 10, rnum = 3, type = {msg, 'ServiceRequest.Register'}, occurrence = optional, opts = []}, #field{name = subscribe, fnum = 11, rnum = 3, type = {msg, 'ServiceRequest.Subscribe'}, occurrence = optional, opts = []}], + opts = []}]; +find_msg_def('ServiceReply.Error') -> [#field{name = code, fnum = 1, rnum = 2, type = int32, occurrence = optional, opts = []}, #field{name = message, fnum = 2, rnum = 3, type = string, occurrence = optional, opts = []}]; +find_msg_def('ServiceReply') -> + [#field{name = packet_id, fnum = 1, rnum = 2, type = uint32, occurrence = optional, opts = []}, + #gpb_oneof{name = reply, rnum = 3, fields = [#field{name = result, fnum = 10, rnum = 3, type = bytes, occurrence = optional, opts = []}, #field{name = error, fnum = 11, rnum = 3, type = {msg, 'ServiceReply.Error'}, occurrence = optional, opts = []}], + opts = []}]; +find_msg_def('ServiceCast.MetricData') -> [#field{name = route_key, fnum = 1, rnum = 2, type = bytes, occurrence = optional, opts = []}, #field{name = metric, fnum = 2, rnum = 3, type = bytes, occurrence = optional, opts = []}]; +find_msg_def('ServiceCast.TopicEvent') -> [#field{name = topic, fnum = 1, rnum = 2, type = string, occurrence = optional, opts = []}, #field{name = content, fnum = 2, rnum = 3, type = bytes, occurrence = optional, opts = []}]; +find_msg_def('ServiceCast') -> + [#gpb_oneof{name = body, rnum = 2, + fields = + [#field{name = topic_event, fnum = 10, rnum = 2, type = {msg, 'ServiceCast.TopicEvent'}, occurrence = optional, opts = []}, #field{name = metric_data, fnum = 11, rnum = 2, type = {msg, 'ServiceCast.MetricData'}, occurrence = optional, opts = []}], + opts = []}]; +find_msg_def(_) -> error. + + +find_enum_def(_) -> error. + + +-spec enum_symbol_by_value(_, _) -> no_return(). +enum_symbol_by_value(E, V) -> erlang:error({no_enum_defs, E, V}). + + +-spec enum_value_by_symbol(_, _) -> no_return(). +enum_value_by_symbol(E, V) -> erlang:error({no_enum_defs, E, V}). + + + +get_service_names() -> []. + + +get_service_def(_) -> error. + + +get_rpc_names(_) -> error. + + +find_rpc_def(_, _) -> error. + + + +-spec fetch_rpc_def(_, _) -> no_return(). +fetch_rpc_def(ServiceName, RpcName) -> erlang:error({no_such_rpc, ServiceName, RpcName}). + + +%% Convert a a fully qualified (ie with package name) service name +%% as a binary to a service name as an atom. +-spec fqbin_to_service_name(_) -> no_return(). +fqbin_to_service_name(X) -> error({gpb_error, {badservice, X}}). + + +%% Convert a service name as an atom to a fully qualified +%% (ie with package name) name as a binary. +-spec service_name_to_fqbin(_) -> no_return(). +service_name_to_fqbin(X) -> error({gpb_error, {badservice, X}}). + + +%% Convert a a fully qualified (ie with package name) service name +%% and an rpc name, both as binaries to a service name and an rpc +%% name, as atoms. +-spec fqbins_to_service_and_rpc_name(_, _) -> no_return(). +fqbins_to_service_and_rpc_name(S, R) -> error({gpb_error, {badservice_or_rpc, {S, R}}}). + + +%% Convert a service name and an rpc name, both as atoms, +%% to a fully qualified (ie with package name) service name and +%% an rpc name as binaries. +-spec service_and_rpc_name_to_fqbins(_, _) -> no_return(). +service_and_rpc_name_to_fqbins(S, R) -> error({gpb_error, {badservice_or_rpc, {S, R}}}). + + +fqbin_to_msg_name(<<"ServiceRequest.Register">>) -> 'ServiceRequest.Register'; +fqbin_to_msg_name(<<"ServiceRequest.Subscribe">>) -> 'ServiceRequest.Subscribe'; +fqbin_to_msg_name(<<"ServiceRequest">>) -> 'ServiceRequest'; +fqbin_to_msg_name(<<"ServiceReply.Error">>) -> 'ServiceReply.Error'; +fqbin_to_msg_name(<<"ServiceReply">>) -> 'ServiceReply'; +fqbin_to_msg_name(<<"ServiceCast.MetricData">>) -> 'ServiceCast.MetricData'; +fqbin_to_msg_name(<<"ServiceCast.TopicEvent">>) -> 'ServiceCast.TopicEvent'; +fqbin_to_msg_name(<<"ServiceCast">>) -> 'ServiceCast'; +fqbin_to_msg_name(E) -> error({gpb_error, {badmsg, E}}). + + +msg_name_to_fqbin('ServiceRequest.Register') -> <<"ServiceRequest.Register">>; +msg_name_to_fqbin('ServiceRequest.Subscribe') -> <<"ServiceRequest.Subscribe">>; +msg_name_to_fqbin('ServiceRequest') -> <<"ServiceRequest">>; +msg_name_to_fqbin('ServiceReply.Error') -> <<"ServiceReply.Error">>; +msg_name_to_fqbin('ServiceReply') -> <<"ServiceReply">>; +msg_name_to_fqbin('ServiceCast.MetricData') -> <<"ServiceCast.MetricData">>; +msg_name_to_fqbin('ServiceCast.TopicEvent') -> <<"ServiceCast.TopicEvent">>; +msg_name_to_fqbin('ServiceCast') -> <<"ServiceCast">>; +msg_name_to_fqbin(E) -> error({gpb_error, {badmsg, E}}). + + +-spec fqbin_to_enum_name(_) -> no_return(). +fqbin_to_enum_name(E) -> error({gpb_error, {badenum, E}}). + + +-spec enum_name_to_fqbin(_) -> no_return(). +enum_name_to_fqbin(E) -> error({gpb_error, {badenum, E}}). + + +get_package_name() -> undefined. + + +%% Whether or not the message names +%% are prepended with package name or not. +uses_packages() -> false. + + +source_basename() -> "service.proto". + + +%% Retrieve all proto file names, also imported ones. +%% The order is top-down. The first element is always the main +%% source file. The files are returned with extension, +%% see get_all_proto_names/0 for a version that returns +%% the basenames sans extension +get_all_source_basenames() -> ["service.proto"]. + + +%% Retrieve all proto file names, also imported ones. +%% The order is top-down. The first element is always the main +%% source file. The files are returned sans .proto extension, +%% to make it easier to use them with the various get_xyz_containment +%% functions. +get_all_proto_names() -> ["service"]. + + +get_msg_containment("service") -> ['ServiceCast', 'ServiceCast.MetricData', 'ServiceCast.TopicEvent', 'ServiceReply', 'ServiceReply.Error', 'ServiceRequest', 'ServiceRequest.Register', 'ServiceRequest.Subscribe']; +get_msg_containment(P) -> error({gpb_error, {badproto, P}}). + + +get_pkg_containment("service") -> undefined; +get_pkg_containment(P) -> error({gpb_error, {badproto, P}}). + + +get_service_containment("service") -> []; +get_service_containment(P) -> error({gpb_error, {badproto, P}}). + + +get_rpc_containment("service") -> []; +get_rpc_containment(P) -> error({gpb_error, {badproto, P}}). + + +get_enum_containment("service") -> []; +get_enum_containment(P) -> error({gpb_error, {badproto, P}}). + + +get_proto_by_msg_name_as_fqbin(<<"ServiceCast.MetricData">>) -> "service"; +get_proto_by_msg_name_as_fqbin(<<"ServiceRequest.Register">>) -> "service"; +get_proto_by_msg_name_as_fqbin(<<"ServiceReply.Error">>) -> "service"; +get_proto_by_msg_name_as_fqbin(<<"ServiceRequest">>) -> "service"; +get_proto_by_msg_name_as_fqbin(<<"ServiceCast.TopicEvent">>) -> "service"; +get_proto_by_msg_name_as_fqbin(<<"ServiceCast">>) -> "service"; +get_proto_by_msg_name_as_fqbin(<<"ServiceRequest.Subscribe">>) -> "service"; +get_proto_by_msg_name_as_fqbin(<<"ServiceReply">>) -> "service"; +get_proto_by_msg_name_as_fqbin(E) -> error({gpb_error, {badmsg, E}}). + + +-spec get_proto_by_service_name_as_fqbin(_) -> no_return(). +get_proto_by_service_name_as_fqbin(E) -> error({gpb_error, {badservice, E}}). + + +-spec get_proto_by_enum_name_as_fqbin(_) -> no_return(). +get_proto_by_enum_name_as_fqbin(E) -> error({gpb_error, {badenum, E}}). + + +-spec get_protos_by_pkg_name_as_fqbin(_) -> no_return(). +get_protos_by_pkg_name_as_fqbin(E) -> error({gpb_error, {badpkg, E}}). + + + +gpb_version_as_string() -> + "4.21.7". + +gpb_version_as_list() -> + [4,21,7]. + +gpb_version_source() -> + "file". diff --git a/apps/efka/src/service/efka_service.erl b/apps/efka/src/service/efka_service.erl new file mode 100644 index 0000000..1857bf7 --- /dev/null +++ b/apps/efka/src/service/efka_service.erl @@ -0,0 +1,145 @@ +%%%------------------------------------------------------------------- +%%% @author anlicheng +%%% @copyright (C) 2025, +%%% @doc +%%% 1. 需要管理服务的整个生命周期,包括: 启动,停止 +%%% 2. 需要监控服务的状态,通过port的方式 +%%% 3. 服务的启动和关闭,需要在更高的层级控制 +%%% @end +%%% Created : 18. 4月 2025 16:50 +%%%------------------------------------------------------------------- +-module(efka_service). +-author("anlicheng"). +-include("efka_tables.hrl"). + +-behaviour(gen_server). + +%% API +-export([start_link/2]). +-export([get_name/1, get_pid/1, attach_channel/2]). +-export([metric_data/3, send_event/3]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). + +-record(state, { + service_id :: binary(), + %% 通道id信息 + channel_pid :: pid() | undefined +}). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec get_name(ServiceId :: binary()) -> atom(). +get_name(ServiceId) when is_binary(ServiceId) -> + list_to_atom("efka_service:" ++ binary_to_list(ServiceId)). + +-spec get_pid(ServiceId :: binary()) -> undefined | pid(). +get_pid(ServiceId) when is_binary(ServiceId) -> + whereis(get_name(ServiceId)). + +-spec metric_data(Pid :: pid(), RouteKey :: binary(), Metric :: binary()) -> ok. +metric_data(Pid, RouteKey, Metric) when is_pid(Pid), is_binary(RouteKey), is_binary(Metric) -> + gen_server:cast(Pid, {metric_data, RouteKey, Metric}). + +-spec send_event(Pid :: pid(), EventType :: integer(), Params :: binary()) -> ok. +send_event(Pid, EventType, Params) when is_pid(Pid), is_integer(EventType), is_binary(Params) -> + gen_server:cast(Pid, {send_event, EventType, Params}). + +-spec attach_channel(pid(), pid()) -> ok | {error, Reason :: binary()}. +attach_channel(Pid, ChannelPid) when is_pid(Pid), is_pid(ChannelPid) -> + gen_server:call(Pid, {attach_channel, ChannelPid}). + +%% @doc Spawns the server and registers the local name (unique) +-spec(start_link(Name :: atom(), Service :: binary()) -> + {ok, Pid :: pid()} | ignore | {error, Reason :: term()}). +start_link(Name, ServiceId) when is_atom(Name), is_binary(ServiceId) -> + gen_server:start_link({local, Name}, ?MODULE, [ServiceId], []). + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +%% @private +%% @doc Initializes the server +-spec(init(Args :: term()) -> + {ok, State :: #state{}} | {ok, State :: #state{}, timeout() | hibernate} | + {stop, Reason :: term()} | ignore). +init([ServiceId]) -> + %% supervisor进程通过exit(ChildPid, shutdown)调用的时候,确保terminate函数被调用 + logger:debug("[efka_service] service_id: ~p, started", [ServiceId]), + {ok, #state{service_id = ServiceId}}. + +%% @private +%% @doc Handling call messages +-spec(handle_call(Request :: term(), From :: {pid(), Tag :: term()}, + State :: #state{}) -> + {reply, Reply :: term(), NewState :: #state{}} | + {reply, Reply :: term(), NewState :: #state{}, timeout() | hibernate} | + {noreply, NewState :: #state{}} | + {noreply, NewState :: #state{}, timeout() | hibernate} | + {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | + {stop, Reason :: term(), NewState :: #state{}}). +%% 绑定channel +handle_call({attach_channel, ChannelPid}, _From, State = #state{channel_pid = OldChannelPid, service_id = ServiceId}) -> + case is_pid(OldChannelPid) andalso is_process_alive(OldChannelPid) of + false -> + erlang:monitor(process, ChannelPid), + logger:debug("[efka_service] service_id: ~p, channel attched", [ServiceId]), + {reply, ok, State#state{channel_pid = ChannelPid}}; + true -> + {reply, {error, <<"channel exists">>}, State} + end; + +handle_call(_Request, _From, State = #state{}) -> + {reply, ok, State}. + +%% @private +%% @doc Handling cast messages +-spec(handle_cast(Request :: term(), State :: #state{}) -> + {noreply, NewState :: #state{}} | + {noreply, NewState :: #state{}, timeout() | hibernate} | + {stop, Reason :: term(), NewState :: #state{}}). +handle_cast({metric_data, RouteKey, Metric}, State = #state{service_id = ServiceId}) -> + logger:debug("[efka_service] metric_data service_id: ~p, route_key: ~p, metric data: ~p", [ServiceId, RouteKey, Metric]), + efka_iot_client:metric_data(RouteKey, Metric), + {noreply, State}; + +handle_cast(_Request, State = #state{}) -> + {noreply, State}. + +%% @private +%% @doc Handling all non call/cast messages +-spec(handle_info(Info :: timeout() | term(), State :: #state{}) -> + {noreply, NewState :: #state{}} | + {noreply, NewState :: #state{}, timeout() | hibernate} | + {stop, Reason :: term(), NewState :: #state{}}). +%% 处理channel进程的退出 +handle_info({'DOWN', _Ref, process, ChannelPid, Reason}, State = #state{channel_pid = ChannelPid, service_id = ServiceId}) -> + logger:debug("[efka_service] service_id: ~p, channel exited: ~p", [ServiceId, Reason]), + {noreply, State#state{channel_pid = undefined}}. + +%% @private +%% @doc This function is called by a gen_server when it is about to +%% terminate. It should be the opposite of Module:init/1 and do any +%% necessary cleaning up. When it returns, the gen_server terminates +%% with Reason. The return value is ignored. +-spec(terminate(Reason :: (normal | shutdown | {shutdown, term()} | term()), + State :: #state{}) -> term()). +terminate(Reason, _State = #state{service_id = ServiceId}) -> + logger:debug("[efka_service] service_id: ~p, terminate with reason: ~p", [ServiceId, Reason]), + ok. + +%% @private +%% @doc Convert process state when code is changed +-spec(code_change(OldVsn :: term() | {down, term()}, State :: #state{}, + Extra :: term()) -> + {ok, NewState :: #state{}} | {error, Reason :: term()}). +code_change(_OldVsn, State = #state{}, _Extra) -> + {ok, State}. + +%%%=================================================================== +%%% Internal functions +%%%=================================================================== diff --git a/apps/efka/src/service/efka_service_channel.erl b/apps/efka/src/service/efka_service_channel.erl new file mode 100644 index 0000000..e5e118f --- /dev/null +++ b/apps/efka/src/service/efka_service_channel.erl @@ -0,0 +1,169 @@ +%%%------------------------------------------------------------------- +%%% @author licheng5 +%%% @copyright (C) 2021, +%%% @doc +%%% +%%% @end +%%% Created : 11. 1月 2021 上午12:17 +%%%------------------------------------------------------------------- +-module(efka_service_channel). +-author("licheng5"). +-include("efka_tables.hrl"). +-include("efka_service_pb.hrl"). + +%% 一级帧类型 +%% REQUEST: 需要响应的请求帧 +%% RESPONSE: 对 REQUEST 的响应帧 +%% CAST: 单向消息帧,不需要响应 +-define(FRAME_REQUEST, 16#01). +-define(FRAME_REPLY, 16#02). +-define(FRAME_CAST, 16#03). + +%% API +-export([init/2]). +-export([websocket_init/1, websocket_handle/2, websocket_info/2, terminate/3]). + +-record(state, { + service_id :: undefined | binary(), + service_pid :: undefined | pid(), + subscribed_topics = sets:new(), + is_registered = false :: boolean() +}). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% 逻辑处理方法 +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +-spec init(term(), term()) -> {cowboy_websocket, term(), term()}. +init(Req, Opts) -> + {cowboy_websocket, Req, Opts}. + +-spec websocket_init(term()) -> {ok, #state{}}. +websocket_init(_State) -> + logger:debug("[efka_service_channel] get a new connection"), + %% 初始状态为true + {ok, #state{}}. + +-spec websocket_handle(term(), #state{}) -> + {reply, term(), #state{}} | {ok, #state{}}. +websocket_handle(ping, State) -> + {reply, pong, State}; + +websocket_handle({binary, <>}, State) -> + Request = efka_service_pb:decode_msg(PacketBin, 'ServiceRequest'), + logger:debug("[efka_service_channel] get request: ~p", [Request]), + handle_request(Request, State); +websocket_handle({binary, <>}, State) -> + Cast = efka_service_pb:decode_msg(PacketBin, 'ServiceCast'), + logger:debug("[efka_service_channel] get cast: ~p", [Cast]), + handle_cast(Cast, State); + +websocket_handle(Info, State) -> + logger:error("[efka_service_channel] get a unknown message: ~p, channel will closed", [Info]), + {ok, State}. + +%% 订阅的消息 +-spec websocket_info(term(), #state{}) -> + {reply, term(), #state{}} | {stop, #state{}} | {ok, #state{}}. +websocket_info({topic_broadcast, Topic, Content}, State = #state{}) -> + Packet = efka_service_pb:encode_msg(#'ServiceCast'{ + body = {topic_event, #'ServiceCast.TopicEvent'{topic = Topic, content = Content}} + }), + logger:debug("[efka_service_channel] will publish topic: ~p", [Topic]), + {reply, {binary, <>}, State}; + +%% service进程关闭 +websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State = #state{service_pid = ServicePid}) -> + logger:debug("[efka_service_channel] container_pid: ~p, exited: ~p", [ServicePid, Reason]), + {stop, State#state{service_pid = undefined}}; + +%% 处理关闭信号 +websocket_info({stop, Reason}, State) -> + logger:debug("[efka_service_channel] the channel will be closed with reason: ~p", [Reason]), + {stop, State}; + +%% 处理其他未知消息 +websocket_info(Info, State) -> + logger:debug("[efka_service_channel] channel get unknown info: ~p", [Info]), + {ok, State}. + +%% 进程关闭事件 +-spec terminate(term(), term(), #state{}) -> ok. +terminate(Reason, _Req, State = #state{service_id = ServiceId, is_registered = IsRegistered}) -> + ok = efka_subscription:unsubscribe_all(self()), + case IsRegistered of + true -> + ok = efka_service_model:change_status(ServiceId, 0); + false -> + ok + end, + logger:debug("[efka_service_channel] channel close with reason: ~p, state is: ~p", [Reason, State]), + ok. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% helper methods +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% 注册, 要建立程序和容器之间的关系 +-spec handle_request(efka_service_pb:'ServiceRequest'(), #state{}) -> {reply, {binary, binary()}, #state{}}. +handle_request(#'ServiceRequest'{packet_id = PacketId, request = {register, #'ServiceRequest.Register'{service_id = ServiceId}}}, State) -> + {ok, ServicePid} = efka_service_sup:start_service(ServiceId), + case efka_service:attach_channel(ServicePid, self()) of + ok -> + erlang:monitor(process, ServicePid), + + %% 更新微服务的状态 + ok = efka_service_model:insert(#service{ + service_id = ServiceId, + container_name = <<>>, + status = ?SERVICE_RUNNING, + meta_data = #{}, + create_ts = efka_util:timestamp(), + update_ts = efka_util:timestamp() + }), + + {reply, {binary, result_reply_packet(PacketId, <<"ok">>)}, + State#state{service_id = ServiceId, service_pid = ServicePid, is_registered = true}}; + {error, Error} -> + logger:warning("[efka_service_channel] service_id: ~p, attach_channel get error: ~p", [ServiceId, Error]), + {reply, {binary, error_reply_packet(PacketId, -1, <<"attach channel failed">>)}, State} + end; + +%% 订阅事件 +handle_request(#'ServiceRequest'{packet_id = PacketId, request = {subscribe, #'ServiceRequest.Subscribe'{topic = Topic}}}, + State = #state{subscribed_topics = SubscribedTopics, is_registered = true}) -> + + case efka_subscription:subscribe(Topic, self()) of + ok -> + Packet = result_reply_packet(PacketId, <<"ok">>), + {reply, {binary, Packet}, State#state{subscribed_topics = sets:add_element(Topic, SubscribedTopics)}}; + {error, Reason} -> + Packet = error_reply_packet(PacketId, -1, Reason), + {reply, {binary, Packet}, State} + end; +handle_request(#'ServiceRequest'{packet_id = PacketId}, State) -> + {reply, {binary, error_reply_packet(PacketId, -1, <<"invalid request">>)}, State}. + +-spec handle_cast(efka_service_pb:'ServiceCast'(), #state{}) -> {ok, #state{}}. +handle_cast(#'ServiceCast'{body = {metric_data, #'ServiceCast.MetricData'{route_key = RouteKey, metric = Metric}}}, + State = #state{service_pid = ServicePid, is_registered = true}) -> + efka_service:metric_data(ServicePid, RouteKey, Metric), + {ok, State}; +handle_cast(#'ServiceCast'{body = _Body}, State) -> + {ok, State}. + +-spec result_reply_packet(integer(), binary()) -> binary(). +result_reply_packet(PacketId, Result) when is_integer(PacketId), is_binary(Result) -> + Reply = efka_service_pb:encode_msg(#'ServiceReply'{ + packet_id = PacketId, + reply = {result, Result} + }), + <>. + +-spec error_reply_packet(integer(), integer(), binary()) -> binary(). +error_reply_packet(PacketId, Code, Message) when is_integer(PacketId), is_integer(Code), is_binary(Message) -> + Reply = efka_service_pb:encode_msg(#'ServiceReply'{ + packet_id = PacketId, + reply = {error, #'ServiceReply.Error'{code = Code, message = Message}} + }), + <>. diff --git a/apps/efka/src/models/service_model.erl b/apps/efka/src/service/efka_service_model.erl similarity index 79% rename from apps/efka/src/models/service_model.erl rename to apps/efka/src/service/efka_service_model.erl index 9e0554f..bc67db0 100644 --- a/apps/efka/src/models/service_model.erl +++ b/apps/efka/src/service/efka_service_model.erl @@ -6,7 +6,7 @@ %%% @end %%% Created : 13. 8月 2025 16:41 %%%------------------------------------------------------------------- --module(service_model). +-module(efka_service_model). -author("anlicheng"). -include("efka_tables.hrl"). @@ -14,6 +14,7 @@ %% API -export([start_link/0]). +-export([insert/1, change_status/2, get_status/1, get_service/1, get_all_services/0, get_running_services/0]). %% gen_server callbacks -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). @@ -29,20 +30,14 @@ %%% API %%%=================================================================== +-spec insert(#service{}) -> ok. insert(Service = #service{}) -> gen_server:call(?SERVER, {insert, Service}). +-spec change_status(binary(), integer()) -> ok | {error, binary()}. change_status(ServiceId, NewStatus) when is_binary(ServiceId), is_integer(NewStatus) -> gen_server:call(?SERVER, {change_status, ServiceId, NewStatus}). --spec set_config(ServiceId :: binary(), ConfigJson :: binary()) -> ok | {error, Reason :: any()}. -set_config(ServiceId, ConfigJson) when is_binary(ServiceId), is_binary(ConfigJson) -> - gen_server:call(?SERVER, {set_config, ServiceId, ConfigJson}). - --spec get_config_json(ServiceId :: binary()) -> error | {ok, ConfigJson :: binary()}. -get_config_json(ServiceId) when is_binary(ServiceId) -> - gen_server:call(?SERVER, {get_config_json, ServiceId}). - -spec get_status(ServiceId :: binary()) -> Status :: integer(). get_status(ServiceId) when is_binary(ServiceId) -> gen_server:call(?SERVER, {get_status, ServiceId}). @@ -55,11 +50,7 @@ get_service(ServiceId) when is_binary(ServiceId) -> get_all_services() -> gen_server:call(?SERVER, get_all_services). --spec get_all_service_ids() -> [ServiceId :: binary()]. -get_all_service_ids() -> - gen_server:call(?SERVER, get_all_service_ids). - --spec get_running_services() -> {ok, [#service{}]} | {error, Reason :: term()}. +-spec get_running_services() -> {ok, [#service{}]}. get_running_services() -> gen_server:call(?SERVER, get_running_services). @@ -81,7 +72,7 @@ start_link() -> init([]) -> {ok, DetsDir} = application:get_env(efka, dets_dir), File = DetsDir ++ "service.dets", - {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, bag}, {keypos, 2}]), + {ok, ?TAB} = dets:open_file(?TAB, [{file, File}, {type, set}, {keypos, 2}]), {ok, #state{}}. %% @private @@ -94,8 +85,18 @@ init([]) -> {noreply, NewState :: #state{}, timeout() | hibernate} | {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | {stop, Reason :: term(), NewState :: #state{}}). -handle_call({insert, Service}, _From, State = #state{}) -> - ok = dets:insert(?TAB, Service), +handle_call({insert, Service = #service{service_id = ServiceId}}, _From, State = #state{}) -> + case dets:lookup(?TAB, ServiceId) of + [] -> + ok = dets:insert(?TAB, Service); + [OldService] -> + NewService = OldService#service{ + meta_data = Service#service.meta_data, + container_name = Service#service.container_name, + update_ts = Service#service.update_ts + }, + ok = dets:insert(?TAB, NewService) + end, {reply, ok, State}; handle_call({change_status, ServiceId, NewStatus}, _From, State = #state{}) -> @@ -108,24 +109,6 @@ handle_call({change_status, ServiceId, NewStatus}, _From, State = #state{}) -> {reply, ok, State} end; -handle_call({set_config, ServiceId, ConfigJson}, _From, State = #state{}) -> - case dets:lookup(?TAB, ServiceId) of - [] -> - {reply, {error, <<"service not found">>}, State}; - [OldService] -> - NewService = OldService#service{config_json = ConfigJson}, - ok = dets:insert(?TAB, NewService), - {reply, ok, State} - end; - -handle_call({get_config_json, ServiceId}, _From, State = #state{}) -> - case dets:lookup(?TAB, ServiceId) of - [] -> - {reply, error, State}; - [#service{config_json = ConfigJson}] -> - {reply, {ok, ConfigJson}, State} - end; - handle_call({get_status, ServiceId}, _From, State = #state{}) -> case dets:lookup(?TAB, ServiceId) of [] -> @@ -141,7 +124,7 @@ handle_call(get_all_services, _From, State = #state{}) -> handle_call(get_running_services, _From, State = #state{}) -> Items = dets:foldl(fun(Record, Acc) -> [Record|Acc] end, [], ?TAB), RunningItems = lists:filter(fun(#service{status = Status}) -> Status =:= 1 end, lists:reverse(Items)), - {reply, RunningItems, State}; + {reply, {ok, RunningItems}, State}; handle_call(_Request, _From, State = #state{}) -> {reply, ok, State}. diff --git a/apps/efka/src/efka_service_sup.erl b/apps/efka/src/service/efka_service_sup.erl similarity index 83% rename from apps/efka/src/efka_service_sup.erl rename to apps/efka/src/service/efka_service_sup.erl index a516296..bbf9f62 100644 --- a/apps/efka/src/efka_service_sup.erl +++ b/apps/efka/src/service/efka_service_sup.erl @@ -39,16 +39,10 @@ start_link() -> %% this function is called by the new process to find out about %% restart strategy, maximum restart frequency and child %% specifications. +-spec init(list()) -> {ok, {supervisor:sup_flags(), [supervisor:child_spec()]}}. init([]) -> SupFlags = #{strategy => one_for_one, intensity => 1000, period => 3600}, - %% 简化逻辑,只启动需要运行的微服务 - {ok, Services} = service_model:get_running_services(), - ServiceIds = lists:map(fun(#service{service_id = ServiceId}) -> ServiceId end, Services), - lager:debug("[efka_service_sup] will start services: ~p", [ServiceIds]), - - Specs = lists:map(fun(ServiceId) -> child_spec(ServiceId) end, Services), - - {ok, {SupFlags, Specs}}. + {ok, {SupFlags, []}}. %%%=================================================================== %%% Internal functions @@ -71,8 +65,7 @@ stop_service(ServiceId) when is_binary(ServiceId) -> supervisor:terminate_child(?MODULE, ChildId), supervisor:delete_child(?MODULE, ChildId). -child_spec(#service{service_id = ServiceId}) when is_binary(ServiceId) -> - child_spec(ServiceId); +-spec child_spec(binary()) -> supervisor:child_spec(). child_spec(ServiceId) when is_binary(ServiceId) -> Name = efka_service:get_name(ServiceId), #{ @@ -82,4 +75,4 @@ child_spec(ServiceId) when is_binary(ServiceId) -> shutdown => 5000, type => worker, modules => ['efka_service'] - }. \ No newline at end of file + }. diff --git a/apps/efka/src/efka_subscription.erl b/apps/efka/src/service/efka_subscription.erl similarity index 54% rename from apps/efka/src/efka_subscription.erl rename to apps/efka/src/service/efka_subscription.erl index 9ea6e7c..12b0d3f 100644 --- a/apps/efka/src/efka_subscription.erl +++ b/apps/efka/src/service/efka_subscription.erl @@ -13,7 +13,7 @@ %% API -export([start_link/0]). --export([subscribe/2, publish/2]). +-export([subscribe/2, unsubscribe_all/1, publish/3, debug_info/0]). -export([match_components/2, is_valid_components/1, of_components/1]). %% gen_server callbacks @@ -34,20 +34,30 @@ }). -record(state, { - subscribers = [] + subscribers = [], + %% qos未1,并且未被消费的消息 + remaining_messages = [] }). %%%=================================================================== %%% API %%%=================================================================== --spec subscribe(Topic :: binary(), SubscriberPid :: pid()) -> no_return(). +-spec subscribe(Topic :: binary(), SubscriberPid :: pid()) -> ok | {error, Reason :: binary()}. subscribe(Topic, SubscriberPid) when is_binary(Topic), is_pid(SubscriberPid) -> - gen_server:cast(?SERVER, {subscribe, Topic, SubscriberPid}). + gen_server:call(?SERVER, {subscribe, Topic, SubscriberPid}). --spec publish(Topic :: binary(), Content :: binary()) -> no_return(). -publish(Topic, Content) when is_binary(Topic), is_binary(Content) -> - gen_server:cast(?SERVER, {publish, Topic, Content}). +-spec publish(Topic :: binary(), Qos :: integer(), Content :: binary()) -> ok. +publish(Topic, Qos, Content) when is_binary(Topic), is_integer(Qos), is_binary(Content) -> + gen_server:cast(?SERVER, {publish, Topic, Qos, Content}). + +-spec unsubscribe_all(pid()) -> ok. +unsubscribe_all(SubscriberPid) when is_pid(SubscriberPid) -> + gen_server:call(?SERVER, {unsubscribe_all, SubscriberPid}). + +-spec debug_info() -> {ok, Info :: map()}. +debug_info() -> + gen_server:call(?SERVER, debug_info). %% @doc Spawns the server and registers the local name (unique) -spec(start_link() -> @@ -77,8 +87,38 @@ init([]) -> {noreply, NewState :: #state{}, timeout() | hibernate} | {stop, Reason :: term(), Reply :: term(), NewState :: #state{}} | {stop, Reason :: term(), NewState :: #state{}}). -handle_call(_Request, _From, State = #state{}) -> - {reply, ok, State}. +%% 同一个SubscriberPid只能订阅同一个topic一次 +handle_call({subscribe, Topic, SubscriberPid}, _From, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) -> + Components = of_components(Topic), + case is_valid_components(Components) of + true -> + case has_subscription(Topic, SubscriberPid, Subscribers) of + true -> + {reply, ok, State}; + false -> + Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)}, + %% 只有首次看到该pid时才建立monitor,避免重复monitor + case has_subscriber_pid(SubscriberPid, Subscribers) of + true -> + ok; + false -> + erlang:monitor(process, SubscriberPid) + end, + %% 处理遗留的消息 + RestRemainingMessages = dispatch_remaining_messages(Sub, RemainingMessages), + {reply, ok, State#state{subscribers = Subscribers ++ [Sub], remaining_messages = RestRemainingMessages}} + end; + false -> + {reply, {error, <<"invalid topic name">>}, State} + end; +handle_call({unsubscribe_all, SubscriberPid}, _From, State = #state{subscribers = Subscribers}) -> + {reply, ok, State#state{subscribers = remove_subscriber_pid(SubscriberPid, Subscribers)}}; +handle_call(debug_info, _From, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) -> + Info = #{ + subscribes => Subscribers, + remaining_messages => RemainingMessages + }, + {reply, {ok, Info}, State}. %% @private %% @doc Handling cast messages @@ -86,29 +126,19 @@ handle_call(_Request, _From, State = #state{}) -> {noreply, NewState :: #state{}} | {noreply, NewState :: #state{}, timeout() | hibernate} | {stop, Reason :: term(), NewState :: #state{}}). -%% 同一个SubscriberPid只能订阅同一个topic一次 -handle_cast({subscribe, Topic, SubscriberPid}, State = #state{subscribers = Subscribers}) -> - Components = of_components(Topic), - case is_valid_components(Components) of - true -> - Sub = #subscriber{topic = Topic, subscriber_pid = SubscriberPid, components = Components, order = order_num(Components)}, - %% 建立到SubscriberPid的monitor,进程退出需要清理订阅 - erlang:monitor(process, SubscriberPid), - - {noreply, State#state{subscribers = Subscribers ++ [Sub]}}; - false -> - {noreply, State} - end; - %% 发布消息 -handle_cast({publish, Topic, Content}, State = #state{subscribers = Subscribers}) -> +handle_cast({publish, Topic, Qos, Content}, State = #state{subscribers = Subscribers, remaining_messages = RemainingMessages}) -> MatchedSubscribers = match_subscribers(Subscribers, Topic), - lists:foreach(fun(#subscriber{subscriber_pid = SubscriberPid}) -> - SubscriberPid ! {topic_broadcast, Topic, Content} - end, MatchedSubscribers), - - lager:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]), - {noreply, State}. + logger:debug("[efka_subscription] topic: ~p, content: ~p, match subscribers: ~p", [Topic, Content, MatchedSubscribers]), + case MatchedSubscribers of + [_|_] -> + broadcast(Topic, Content, MatchedSubscribers), + {noreply, State}; + [] when Qos =:= 0 -> + {noreply, State}; + [] -> + {noreply, State#state{remaining_messages = [{Topic, Content}|RemainingMessages]}} + end. %% @private %% @doc Handling all non call/cast messages @@ -117,12 +147,12 @@ handle_cast({publish, Topic, Content}, State = #state{subscribers = Subscribers} {noreply, NewState :: #state{}, timeout() | hibernate} | {stop, Reason :: term(), NewState :: #state{}}). handle_info({'DOWN', _Ref, process, SubscriberPid, Reason}, State = #state{subscribers = Subscribers}) -> - lager:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]), - NSubscribers = lists:filter(fun(#subscriber{subscriber_pid = Pid0}) -> SubscriberPid /= Pid0 end, Subscribers), + logger:debug("[efka_subscription] subscriber: ~p, down with reason: ~p", [SubscriberPid, Reason]), + NSubscribers = remove_subscriber_pid(SubscriberPid, Subscribers), {noreply, State#state{subscribers = NSubscribers}}; handle_info(Info, State = #state{}) -> - lager:debug("[efka_subscription] get unknown info: ~p", [Info]), + logger:debug("[efka_subscription] get unknown info: ~p", [Info]), {noreply, State}. %% @private @@ -164,6 +194,17 @@ match_subscribers(Subscribers, Topic) when is_list(Subscribers), is_binary(Topic contain_channel(Pid, Subscribers) when is_pid(Pid), is_list(Subscribers) -> lists:search(fun(#subscriber{subscriber_pid = Pid0}) -> Pid == Pid0 end, Subscribers) /= false. +-spec has_subscriber_pid(pid(), [#subscriber{}]) -> boolean(). +has_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) -> + lists:any(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =:= SubscriberPid0 end, Subscribers). + +-spec has_subscription(binary(), pid(), [#subscriber{}]) -> boolean(). +has_subscription(Topic, SubscriberPid, Subscribers) + when is_binary(Topic), is_pid(SubscriberPid), is_list(Subscribers) -> + lists:any(fun(#subscriber{topic = Topic0, subscriber_pid = SubscriberPid0}) -> + Topic =:= Topic0 andalso SubscriberPid =:= SubscriberPid0 + end, Subscribers). + %% 开始对比订阅的topic和发布的topic的Components信息 %% *表示单级匹配,+表示多级匹配;+只能出现一次,并且只能在末尾 -spec match_components(list(), list()) -> boolean(). @@ -184,6 +225,7 @@ match_components(_, _, _) -> of_components(Topic) when is_binary(Topic) -> binary:split(Topic, <<$/>>, [global]). +-spec is_valid_components([binary()]) -> boolean(). is_valid_components([]) -> true; is_valid_components([<<$+>>|T]) -> @@ -201,4 +243,28 @@ order_num([<<$*>>|_]) -> order_num([<<$+>>|_]) -> 3; order_num([_|Tail]) -> - order_num(Tail). \ No newline at end of file + order_num(Tail). + +-spec broadcast(binary(), binary(), [#subscriber{}]) -> ok. +broadcast(Topic, Content, MatchedSubscribers) -> + lists:foreach(fun(#subscriber{subscriber_pid = SubscriberPid}) -> + SubscriberPid ! {topic_broadcast, Topic, Content} + end, MatchedSubscribers). + +-spec remove_subscriber_pid(pid(), [#subscriber{}]) -> [#subscriber{}]. +remove_subscriber_pid(SubscriberPid, Subscribers) when is_pid(SubscriberPid), is_list(Subscribers) -> + lists:filter(fun(#subscriber{subscriber_pid = SubscriberPid0}) -> SubscriberPid =/= SubscriberPid0 end, Subscribers). + +-spec dispatch_remaining_messages(Subscriber :: #subscriber{}, RemainingMessages :: list()) -> RestRemainingMessages :: list(). +dispatch_remaining_messages(#subscriber{subscriber_pid = SubscriberPid, components = Components}, RemainingMessages) when is_list(RemainingMessages) -> + %% 处理遗留的消息 + lists:foldl(fun({Topic0, Content0}, Acc) -> + Components0 = of_components(Topic0), + case match_components(Components, Components0) of + true -> + SubscriberPid ! {topic_broadcast, Topic0, Content0}, + Acc; + false -> + [{Topic0, Content0}|Acc] + end + end, [], RemainingMessages). diff --git a/apps/efka/src/tests/efka_iot_outbox_test.erl b/apps/efka/src/tests/efka_iot_outbox_test.erl new file mode 100644 index 0000000..d4ef530 --- /dev/null +++ b/apps/efka/src/tests/efka_iot_outbox_test.erl @@ -0,0 +1,315 @@ +%%%------------------------------------------------------------------- +%%% @doc +%%% efka_iot_outbox 的手动模拟测试模块。 +%%% +%%% 该模块基于 gen_server 实现,用两个定时 tick 模拟一边写入、一边读取: +%%% - write_tick 每次向 efka_iot_outbox 写入一条 packet。 +%%% - read_tick 每次从 efka_iot_outbox 读取并 ack 一条 packet。 +%%% - 默认 read_interval = write_interval * 2,因此读取速度是写入速度的 +%%% 50%。 +%%% - 读取到的数据会通过 logger:debug/2 输出。 +%%% +%%% 默认使用独立临时 dir,避免污染真实运行目录。也可以通过 start_link/1 +%%% 传入 {dir, Dir} 指定 outbox 目录。 +%%% +%%% 示例: +%%%
+%%% efka_iot_outbox_test:start_link([
+%%%     {dir, "/private/tmp/efka_iot_outbox_test"},
+%%%     {write_interval, 100},
+%%%     {read_interval, 200},
+%%%     {max_writes, 1000}
+%%% ]).
+%%% efka_iot_outbox_test:stats().
+%%% efka_iot_outbox_test:stop().
+%%% 
+%%% @end +%%%------------------------------------------------------------------- +-module(efka_iot_outbox_test). + +-behaviour(gen_server). + +-export([start/0, start/1, start_link/0, start_link/1, stop/0, stats/0]). +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). +-export([test/0]). + +-define(SERVER, ?MODULE). +-define(DEFAULT_WRITE_INTERVAL, 100). +-define(DEFAULT_SEGMENT_RECORD_LIMIT, 2000). +-define(DEFAULT_MAX_SEGMENTS, 5). +-define(DEFAULT_MAX_RECORD_BYTES, 16 * 1024 * 1024). + +-record(state, { + outbox :: efka_iot_outbox:outbox(), + dir :: file:filename_all(), + write_interval = ?DEFAULT_WRITE_INTERVAL :: pos_integer(), + read_interval = ?DEFAULT_WRITE_INTERVAL * 2 :: pos_integer(), + max_writes = infinity :: pos_integer() | infinity, + write_timer = undefined :: undefined | reference(), + read_timer = undefined :: undefined | reference(), + write_count = 0 :: non_neg_integer(), + appended_count = 0 :: non_neg_integer(), + read_count = 0 :: non_neg_integer(), + dropped_count = 0 :: non_neg_integer(), + error_count = 0 :: non_neg_integer() +}). + +-type option() :: + {dir, file:filename_all()} | + {write_interval, pos_integer()} | + {read_interval, pos_integer()} | + {max_writes, pos_integer() | infinity} | + {segment_record_limit, pos_integer()} | + {max_segments, pos_integer()} | + {max_record_bytes, pos_integer()}. + + +test() -> + efka_iot_outbox_test:start_link([ + {write_interval, 10}, + {read_interval, 20}, + {max_writes, 100_0000} + ]). + +-spec start() -> {ok, pid()} | {error, term()}. +start() -> + start([]). + +-spec start([option()] | map()) -> {ok, pid()} | {error, term()}. +start(Options) -> + gen_server:start({local, ?SERVER}, ?MODULE, Options, []). + +-spec start_link() -> {ok, pid()} | {error, term()}. +start_link() -> + start_link([]). + +-spec start_link([option()] | map()) -> {ok, pid()} | {error, term()}. +start_link(Options) -> + gen_server:start_link({local, ?SERVER}, ?MODULE, Options, []). + +-spec stop() -> ok. +stop() -> + gen_server:stop(?SERVER). + +-spec stats() -> map(). +stats() -> + gen_server:call(?SERVER, stats). + +-spec init([option()] | map()) -> {ok, #state{}} | {stop, term()}. +init(Options0) -> + process_flag(trap_exit, true), + Options = normalize_options(Options0), + WriteInterval = positive_option(write_interval, Options, ?DEFAULT_WRITE_INTERVAL), + ReadInterval = positive_option(read_interval, Options, WriteInterval * 2), + SegmentRecordLimit = positive_option(segment_record_limit, Options, ?DEFAULT_SEGMENT_RECORD_LIMIT), + MaxSegments = positive_option(max_segments, Options, ?DEFAULT_MAX_SEGMENTS), + MaxRecordBytes = positive_option(max_record_bytes, Options, ?DEFAULT_MAX_RECORD_BYTES), + MaxWrites = max_writes_option(Options), + + Dir = "/private/tmp/efka_outbox/", + ok = filelib:ensure_dir(filename:join(Dir, "dummy")), + OutboxOptions = #{ + dir => Dir, + segment_record_limit => SegmentRecordLimit, + max_segments => MaxSegments, + max_record_bytes => MaxRecordBytes + }, + case efka_iot_outbox:open(OutboxOptions) of + {ok, Outbox} -> + State0 = #state{ + outbox = Outbox, + dir = Dir, + write_interval = WriteInterval, + read_interval = ReadInterval, + max_writes = MaxWrites + }, + State1 = schedule_write(State0, 0), + State2 = schedule_read(State1, ReadInterval), + {ok, State2}; + {error, Reason} -> + {stop, Reason} + end. + +-spec handle_call(term(), gen_server:from(), #state{}) -> + {reply, term(), #state{}}. +handle_call(stats, _From, State) -> + {reply, state_stats(State), State}; +handle_call(_Request, _From, State) -> + {reply, {error, unknown_call}, State}. + +-spec handle_cast(term(), #state{}) -> {noreply, #state{}}. +handle_cast(_Request, State) -> + {noreply, State}. + +-spec handle_info(term(), #state{}) -> {noreply, #state{}}. +handle_info(write_tick, State0 = #state{write_timer = WriteTimer}) -> + State1 = State0#state{write_timer = undefined}, + State2 = case can_write(State1) of + true -> + append_one(State1); + false -> + State1 + end, + State3 = case can_write(State2) of + true -> + schedule_write(State2, State2#state.write_interval); + false -> + State2 + end, + _ = WriteTimer, + {noreply, State3}; +handle_info(read_tick, State0 = #state{read_timer = ReadTimer}) -> + State1 = State0#state{read_timer = undefined}, + State2 = read_one(State1), + State3 = schedule_read(State2, State2#state.read_interval), + _ = ReadTimer, + {noreply, State3}; +handle_info(_Info, State) -> + {noreply, State}. + +-spec terminate(term(), #state{}) -> ok. +terminate(_Reason, #state{outbox = Outbox, write_timer = WriteTimer, read_timer = ReadTimer}) -> + cancel_timer(WriteTimer), + cancel_timer(ReadTimer), + efka_iot_outbox:close(Outbox), + ok. + +-spec code_change(term(), #state{}, term()) -> {ok, #state{}}. +code_change(_OldVsn, State, _Extra) -> + {ok, State}. + +-spec append_one(#state{}) -> #state{}. +append_one(State = #state{outbox = Outbox, write_count = WriteCount}) -> + PacketId = WriteCount + 1, + Packet = term_to_binary(#{ + source => efka_iot_outbox_test, + packet_id => PacketId, + monotonic_time => erlang:monotonic_time(millisecond) + }), + case efka_iot_outbox:append(Packet, Outbox) of + {ok, NOutbox} -> + State#state{ + outbox = NOutbox, + write_count = PacketId, + appended_count = State#state.appended_count + 1 + }; + {dropped, capacity_reached, NOutbox} -> + logger:warning("[efka_iot_outbox_test] outbox capacity reached, packet_id: ~p", [PacketId]), + State#state{ + outbox = NOutbox, + write_count = PacketId, + dropped_count = State#state.dropped_count + 1 + }; + {error, Reason} -> + logger:warning("[efka_iot_outbox_test] append failed, packet_id: ~p, reason: ~p", [PacketId, Reason]), + State#state{ + write_count = PacketId, + error_count = State#state.error_count + 1 + } + end. + +-spec read_one(#state{}) -> #state{}. +read_one(State = #state{outbox = Outbox}) -> + case efka_iot_outbox:next(Outbox) of + {ok, Seq, Packet} -> + logger:debug("[efka_iot_outbox_test] read seq: ~p, packet: ~p", [Seq, decode_packet(Packet)]), + case efka_iot_outbox:ack(Seq, Outbox) of + {ok, NOutbox} -> + State#state{ + outbox = NOutbox, + read_count = State#state.read_count + 1 + }; + {error, Reason} -> + logger:warning("[efka_iot_outbox_test] ack failed, seq: ~p, reason: ~p", [Seq, Reason]), + State#state{error_count = State#state.error_count + 1} + end; + eof -> + State; + {error, Reason} -> + logger:warning("[efka_iot_outbox_test] read failed, reason: ~p", [Reason]), + State#state{error_count = State#state.error_count + 1} + end. + +-spec schedule_write(#state{}, non_neg_integer()) -> #state{}. +schedule_write(State, Delay) -> + Ref = erlang:send_after(Delay, self(), write_tick), + State#state{write_timer = Ref}. + +-spec schedule_read(#state{}, non_neg_integer()) -> #state{}. +schedule_read(State, Delay) -> + Ref = erlang:send_after(Delay, self(), read_tick), + State#state{read_timer = Ref}. + +-spec cancel_timer(undefined | reference()) -> ok. +cancel_timer(undefined) -> + ok; +cancel_timer(Ref) -> + _ = erlang:cancel_timer(Ref), + ok. + +-spec can_write(#state{}) -> boolean(). +can_write(#state{max_writes = infinity}) -> + true; +can_write(#state{write_count = WriteCount, max_writes = MaxWrites}) -> + WriteCount < MaxWrites. + +-spec state_stats(#state{}) -> map(). +state_stats(#state{ + dir = Dir, + write_interval = WriteInterval, + read_interval = ReadInterval, + max_writes = MaxWrites, + write_count = WriteCount, + appended_count = AppendedCount, + read_count = ReadCount, + dropped_count = DroppedCount, + error_count = ErrorCount +}) -> + #{ + dir => Dir, + write_interval => WriteInterval, + read_interval => ReadInterval, + max_writes => MaxWrites, + write_count => WriteCount, + appended_count => AppendedCount, + read_count => ReadCount, + dropped_count => DroppedCount, + error_count => ErrorCount, + pending_count => AppendedCount - ReadCount + }. + +-spec normalize_options([option()] | map()) -> map(). +normalize_options(Options) when is_map(Options) -> + Options; +normalize_options(Options) when is_list(Options) -> + maps:from_list(Options). + +-spec positive_option(atom(), map(), pos_integer()) -> pos_integer(). +positive_option(Key, Options, Default) -> + case maps:get(Key, Options, Default) of + Value when is_integer(Value), Value > 0 -> + Value; + Value -> + error({invalid_option, Key, Value}) + end. + +-spec max_writes_option(map()) -> pos_integer() | infinity. +max_writes_option(Options) -> + case maps:get(max_writes, Options, infinity) of + infinity -> + infinity; + Value when is_integer(Value), Value > 0 -> + Value; + Value -> + error({invalid_option, max_writes, Value}) + end. + +-spec decode_packet(binary()) -> term(). +decode_packet(Packet) -> + try binary_to_term(Packet, [safe]) of + Term -> + Term + catch + error:_ -> + Packet + end. diff --git a/config/sys.config b/config/sys.config index f49e156..33b3bb7 100644 --- a/config/sys.config +++ b/config/sys.config @@ -1,60 +1,60 @@ [ {efka, [ - {root_dir, "/usr/local/code/efka"}, + {dets_dir, "/var/lib/efka/dets/"}, - {dets_dir, "/tmp/db/"}, - - {tcp_server, [ - {port, 18088} + {websocket_server, [ + {port, 18080}, + {acceptors, 10}, + {max_connections, 1024}, + {backlog, 256} ]}, - {tls_server, [ + {iot_server, [ {host, "localhost"}, - {port, 443} + {tls_port, 1443}, + {udp_port, 18080} + ]}, + + {heartbeat, [ + {interval, 5000} ]}, {auth, [ {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"}, - {username, "test"}, - {salt, "salt2345"}, - {token, "token124"} + {token, "zpxlkvmqwnbghytrujsdieofazxcvbnm"} ]} ]}, - %% 系统日志配置,系统日志为lager, 支持日志按日期自动分割 - {lager, [ - {colored, true}, - %% Whether to write a crash log, and where. Undefined means no crash logger. - {crash_log, "trade_hub.crash.log"}, - %% Maximum size in bytes of events in the crash log - defaults to 65536 - {crash_log_msg_size, 65536}, - %% Maximum size of the crash log in bytes, before its rotated, set - %% to 0 to disable rotation - default is 0 - {crash_log_size, 10485760}, - %% What time to rotate the crash log - default is no time - %% rotation. See the README for a description of this format. - {crash_log_date, "$D0"}, - %% Number of rotated crash logs to keep, 0 means keep only the - %% current one - default is 0 - {crash_log_count, 5}, - %% Whether to redirect error_logger messages into lager - defaults to true - {error_logger_redirect, true}, + {docker, [ + {root_dir, "/var/lib/efka/docker/"} + ]}, - %% How big the gen_event mailbox can get before it is switched into sync mode - {async_threshold, 20}, - %% Switch back to async mode, when gen_event mailbox size decrease from `async_threshold' - %% to async_threshold - async_threshold_window - {async_threshold_window, 5}, + %% 系统日志配置,使用 OTP logger + {kernel, [ + %% 设置 Logger 的 primary log level + {logger_level, debug}, + {logger, [ + {handler, default, logger_std_h, + #{ + level => debug, + formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}} + } + }, + + {handler, disk, logger_disk_log_h, + #{ + level => debug, + config => #{ + file => "log/debug.log", + max_no_files => 10, + max_no_bytes => 524288000 + }, + formatter => {logger_formatter, #{template => [time, " [", level, "] ", msg, "\n"]}} + } + } - {handlers, [ - %% debug | info | warning | error, 日志级别 - {lager_console_backend, debug}, - {lager_file_backend, [{file, "debug.log"}, {level, debug}, {size, 314572800}]}, - {lager_file_backend, [{file, "notice.log"}, {level, notice}, {size, 314572800}]}, - {lager_file_backend, [{file, "error.log"}, {level, error}, {size, 314572800}]}, - {lager_file_backend, [{file, "info.log"}, {level, info}, {size, 314572800}]} ]} - ]} -]. \ No newline at end of file + +]. diff --git a/config/vm.args b/config/vm.args index 33441c1..06d9f87 100644 --- a/config/vm.args +++ b/config/vm.args @@ -3,7 +3,7 @@ -setcookie efka_cookie -kernel start_group false --mnesia dir '"/usr/local/var/mnesia/efka"' +-mnesia dir '"/var/lib/efka/mnesia"' -mnesia dump_log_write_threshold 5000 -mnesia dc_dump_limit 40 diff --git a/docs/container_command_docker_json.md b/docs/container_command_docker_json.md new file mode 100644 index 0000000..1c18b56 --- /dev/null +++ b/docs/container_command_docker_json.md @@ -0,0 +1,803 @@ +# IOT 容器命令到 Docker JSON 的转换说明 + +本文档描述 `efka` 收到 `iot` 下发的容器管理 command 后,接受的 Erlang map 格式,以及 deploy 命令中 `create` 参数如何转换成 Docker Engine API 接收的 JSON。 + +对应代码: + +- command 接收入口:[src/iot/efka_iot_client.erl](/usr/local/code/cloudkit/efka/apps/efka/src/iot/efka_iot_client.erl:177) +- 部署任务管理:[docker_deploy_manager.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deploy_manager.erl:36) +- 部署执行:[docker_deployer.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_deployer.erl:39) +- Docker JSON 构造:[docker_container_builder.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_container_builder.erl:14) +- Docker API 调用:[docker_commands.erl](/usr/local/code/cloudkit/efka/apps/docker/src/docker_commands.erl:36) + +## 1. 协议入口 + +`iot` 通过 TLS 长连接向 `efka` 下发容器命令: + +```erlang +{<<"command">>, Ref, {<<"container">>, CommandMap}} +``` + +`efka` 执行后回复: + +```erlang +{<<"command_response">>, Ref, {<<"container">>, Reply}} +``` + +`Reply` 取值: + +```erlang +<<"ok">> +{<<"ok">>, Result} +{<<"error">>, Reason} +``` + +`Ref` 是 `crypto:strong_rand_bytes(16)` 生成的 16 字节 binary。网络帧只使用 `binary_to_term(PacketBin, [safe])` 可解码的 safe term;协议 label、业务 label、map key 和 action 都使用 binary。 + +只有 `efka_iot_client` 处于 `activated` 状态时,容器命令才会正常执行;处于非 activated 状态时会返回错误。 + +## 2. 容器命令 map + +`efka_iot_client` 收到网络协议里的 binary-key `CommandMap` 后,直接按 binary key 和 binary action 做函数参数匹配,不再做整包 atom-key 转换。 + +- command 顶层、`target`、deploy `params`、`create` 中间结构都使用 binary key。 +- `<<"action">>` 的取值使用 binary:`<<"list">>`、`<<"deploy">>`、`<<"start">>`、`<<"stop">>`、`<<"kill">>`、`<<"remove">>`、`<<"config">>`。 +- `undefined` 作为 Erlang 已有 atom 可以直接出现在 safe term 中;协议不再使用额外的哨兵 binary 表示缺省值。 + +下面各小节描述的是 `efka` 直接接收和处理的 map 格式。 + +### list + +```erlang +#{<<"action">> => <<"list">>} +``` + +执行: + +```erlang +docker_commands:get_containers() +``` + +Docker API: + +```http +GET /containers/json?all=true +``` + +### deploy + +```erlang +#{ + <<"action">> => <<"deploy">>, + <<"task_id">> => TaskId, + <<"params">> => Params +} +``` + +执行: + +```erlang +docker_deploy_manager:deploy(TaskId, Params) +``` + +`deploy/2` 会启动独立部署进程,HTTP command response 只表示部署任务是否成功启动。实际部署过程和结果通过 `task_event` 消息流上报给 `iot`。 + +### start + +```erlang +#{ + <<"action">> => <<"start">>, + <<"target">> => Target +} +``` + +执行: + +```erlang +docker_commands:start_container(ContainerNameOrId) +``` + +Docker API: + +```http +POST /containers/{name_or_id}/start +``` + +### stop + +```erlang +#{ + <<"action">> => <<"stop">>, + <<"target">> => Target, + <<"timeout_seconds">> => TimeoutSeconds +} +``` + +执行: + +```erlang +docker_commands:stop_container(ContainerNameOrId, TimeoutSeconds) +``` + +Docker API: + +```http +POST /containers/{name_or_id}/stop?t={TimeoutSeconds} +``` + +### kill + +```erlang +#{ + <<"action">> => <<"kill">>, + <<"target">> => Target, + <<"signal">> => Signal +} +``` + +执行: + +```erlang +docker_commands:kill_container(ContainerNameOrId, Signal) +``` + +### remove + +```erlang +#{ + <<"action">> => <<"remove">>, + <<"target">> => Target, + <<"force">> => Force, + <<"remove_volumes">> => RemoveVolumes +} +``` + +执行: + +```erlang +docker_commands:remove_container(ContainerNameOrId, Force, RemoveVolumes) +``` + +### config + +```erlang +#{ + <<"action">> => <<"config">>, + <<"target">> => Target, + <<"config">> => Config +} +``` + +执行: + +```erlang +docker_helper:update_container_config(ContainerNameOrId, iolist_to_binary(Config)) +``` + +该命令不会调用 Docker API,只会更新 `efka` 主机上对应容器目录里的 `service.conf`。 + +## 3. Target 解析规则 + +`target` 是 map: + +```erlang +#{ + <<"name">> => ContainerName, + <<"id">> => ContainerId +} +``` + +解析规则: + +- 优先使用 `name`。 +- `name` 为空时使用 `id`。 +- `name` 和 `id` 都为空时会触发匹配错误。 + +## 4. deploy params 格式 + +`deploy` 的 `params` 必须包含: + +```erlang +#{ + <<"container_name">> => ContainerName, + <<"create">> => Create +} +``` + +字段说明: + +| 字段 | 类型 | 说明 | +| --- | --- | --- | +| `container_name` | binary | 容器名称。用于容器创建 URL 的 `name` 参数,也会注入环境变量 `CONTAINER_NAME`。 | +| `create` | map | Docker create options 的中间结构,由 `iot` 侧构造,`efka` 侧补丁后转为 Docker JSON。 | + +`docker_deploy_manager` 会根据 `container_name` 按系统默认规则确保容器目录存在。目录固定为: + +```text +docker.root_dir/container_name/ +``` + +即使旧客户端在 `params` 中携带 `container_dir`,当前 deploy 流程也会忽略它,避免 HTTP 调用方控制 efka 主机上的写入目录。 + +## 5. create 中间结构 + +`create` 的结构: + +```erlang +#{ + <<"config">> => ContainerConfig, + <<"host_config">> => HostConfig, + <<"networking_config">> => NetworkingConfig +} +``` + +这三个 map 会被 `docker_container_builder:build_options/3` 转成 Docker Engine API JSON。 + +## 6. efka 自动补丁规则 + +在转 Docker JSON 前,`efka` 会先执行补丁: + +### 环境变量 + +向 `create.config.env` 前置注入: + +```erlang +<<"CONTAINER_NAME=", ContainerName/binary>> +``` + +如果原 env 列表已经包含完全相同的值,则不会重复添加。 + +### 配置文件 volume + +向 `create.config.volumes` 前置注入容器内配置路径: + +```erlang +<<"/usr/local/etc/service.conf">> +``` + +### 配置文件 bind + +向 `create.host_config.binds` 前置注入宿主机配置文件映射: + +```erlang +<> +``` + +其中 `ConfigFile` 是当前容器目录下的: + +```text +service.conf +``` + +如果列表里已存在完全相同的 bind,则不会重复添加。 + +## 7. Docker JSON 顶层结构 + +`docker_commands:create_container/2` 最终调用: + +```http +POST /containers/create?name={container_name} +Content-Type: application/json +``` + +请求 body 来自: + +```erlang +Options = docker_container_builder:build_options(ContainerName, ContainerDir, Create), +Body = iolist_to_binary(json:encode(Options)) +``` + +最终 JSON 顶层字段: + +```json +{ + "Image": "...", + "Cmd": [], + "Entrypoint": [], + "Env": [], + "Labels": {}, + "Volumes": {}, + "User": "", + "WorkingDir": "", + "Hostname": "", + "ExposedPorts": {}, + "NetworkingConfig": {}, + "Healthcheck": {}, + "HostConfig": {} +} +``` + +空列表或空 map 字段有的会保留,有的会被省略到 `{}`,具体见下面映射规则。 + +## 8. create.config 到 Docker JSON + +| 中间字段 | Docker JSON 字段 | 转换规则 | +| --- | --- | --- | +| `image` | `Image` | 转成 binary;缺省为 `""`。部署流程拉镜像时会对 image 补 `:latest`,但 create JSON 使用传入值。 | +| `cmd` | `Cmd` | 列表元素逐个转 binary。 | +| `entrypoint` | `Entrypoint` | 列表元素逐个转 binary。 | +| `env` | `Env` | 列表元素逐个转 binary,并自动注入 `CONTAINER_NAME=...`。 | +| `labels` | `Labels` | key/value 都转 binary。 | +| `volumes` | `Volumes` | 转成 Docker 要求的 object,key 是容器内路径,value 是 `{}`。 | +| `user` | `User` | 转成 binary;缺省为 `""`。 | +| `working_dir` | `WorkingDir` | 转成 binary;缺省为 `""`。 | +| `hostname` | `Hostname` | 转成 binary;缺省为 `""`。 | +| `exposed_ports` | `ExposedPorts` | 转成 Docker 端口 object。 | +| `healthcheck` | `Healthcheck` | 转成 Docker Healthcheck object;未传时为 `{}`。 | + +### Volumes + +输入: + +```erlang +[<<"/data">>, <<"/usr/local/etc/service.conf">>] +``` + +输出 JSON: + +```json +{ + "Volumes": { + "/data": {}, + "/usr/local/etc/service.conf": {} + } +} +``` + +### ExposedPorts + +输入: + +```erlang +[ + #{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>}, + #{<<"container_port">> => 53, <<"protocol">> => <<"udp">>} +] +``` + +输出 JSON: + +```json +{ + "ExposedPorts": { + "80/tcp": {}, + "53/udp": {} + } +} +``` + +规则: + +- `protocol` 为空或 `<<"tcp">>` 时输出 `tcp`。 +- 其他 protocol 原样输出。 + +### Healthcheck + +该字段由 iot 校验后下发。`test` 必须是 binary list,`interval_ns`、`timeout_ns` 和 `retries` 必须是非负整数;efka 只按收到的中间 map 转成 Docker JSON 字段。 + +输入: + +```erlang +#{ + <<"test">> => [<<"CMD-SHELL">>, <<"curl -f http://localhost || exit 1">>], + <<"interval_ns">> => 30000000000, + <<"timeout_ns">> => 10000000000, + <<"retries">> => 3 +} +``` + +输出 JSON: + +```json +{ + "Healthcheck": { + "Test": ["CMD-SHELL", "curl -f http://localhost || exit 1"], + "Interval": 30000000000, + "Timeout": 10000000000, + "Retries": 3 + } +} +``` + +## 9. create.host_config 到 Docker JSON + +`host_config` 会被合并到 Docker JSON 的 `HostConfig` 字段。 + +| 中间字段 | Docker JSON 字段 | 转换规则 | +| --- | --- | --- | +| `binds` | `HostConfig.Binds` | 列表元素逐个转 binary,并自动注入 `service.conf` bind。空列表时省略。 | +| `network_mode` | `HostConfig.NetworkMode` | 空 binary 时省略。 | +| `restart_policy` | `HostConfig.RestartPolicy` | 转成 `Name` 和可选 `MaximumRetryCount`。 | +| `privileged` | `HostConfig.Privileged` | 只有 true 时输出。false 时省略。 | +| `cap_add` / `cap_drop` | `HostConfig.CapAdd` / `HostConfig.CapDrop` | 两者都为空时省略;只要一个非空,两个字段都会输出。 | +| `devices` | `HostConfig.Devices` | 转成 Docker device object 列表。 | +| `memory` | `HostConfig.Memory` | 0 时省略。 | +| `memory_reservation` | `HostConfig.MemoryReservation` | 0 时省略。 | +| `nano_cpus` | `HostConfig.NanoCpus` | 0 时省略。 | +| `cpu_shares` | `HostConfig.CpuShares` | 0 时省略。 | +| `port_bindings` | `HostConfig.PortBindings` | 转成 Docker 端口绑定 object;空列表时省略。 | +| `ulimits` | `HostConfig.Ulimits` | 空列表时省略。 | +| `tmpfs` | `HostConfig.Tmpfs` | 空 map 时省略。 | +| `sysctls` | `HostConfig.Sysctls` | 空 map 时省略。 | +| `extra_hosts` | `HostConfig.ExtraHosts` | 空列表时省略。 | + +### Binds + +输入: + +```erlang +[ + <<"/host/data:/data">>, + <<"/host/log:/var/log:ro">> +] +``` + +efka 自动补丁后输出: + +```json +{ + "HostConfig": { + "Binds": [ + "/path/to/container/service.conf:/usr/local/etc/service.conf", + "/host/data:/data", + "/host/log:/var/log:ro" + ] + } +} +``` + +### RestartPolicy + +输入: + +```erlang +#{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0} +``` + +输出: + +```json +{ + "HostConfig": { + "RestartPolicy": { + "Name": "always" + } + } +} +``` + +输入: + +```erlang +#{<<"name">> => <<"on-failure">>, <<"maximum_retry_count">> => 3} +``` + +输出: + +```json +{ + "HostConfig": { + "RestartPolicy": { + "Name": "on-failure", + "MaximumRetryCount": 3 + } + } +} +``` + +### Devices + +输入: + +```erlang +[ + #{ + <<"path_on_host">> => <<"/dev/ttyUSB0">>, + <<"path_in_container">> => <<"/dev/ttyUSB0">>, + <<"cgroup_permissions">> => <<"rwm">> + } +] +``` + +输出: + +```json +{ + "HostConfig": { + "Devices": [ + { + "PathOnHost": "/dev/ttyUSB0", + "PathInContainer": "/dev/ttyUSB0", + "CgroupPermissions": "rwm" + } + ] + } +} +``` + +`cgroup_permissions` 为空时默认输出 `rwm`。 + +### PortBindings + +输入: + +```erlang +[ + #{<<"host_ip">> => <<>>, <<"host_port">> => 8080, <<"container_port">> => 80, <<"protocol">> => <<"tcp">>}, + #{<<"host_ip">> => <<>>, <<"host_port">> => 443, <<"container_port">> => 443, <<"protocol">> => <<"tcp">>} +] +``` + +输出: + +```json +{ + "HostConfig": { + "PortBindings": { + "80/tcp": [ + {"HostIp": "", "HostPort": "8080"} + ], + "443/tcp": [ + {"HostIp": "", "HostPort": "443"} + ] + } + } +} +``` + +规则: + +- key 使用容器端口和协议组成:`container_port/protocol`。 +- `protocol` 为空或 `tcp` 时输出 `tcp`。 +- `HostPort` 按 Docker API 要求输出为字符串。 + +### 资源限制 + +输入: + +```erlang +#{ + <<"memory">> => 536870912, + <<"memory_reservation">> => 268435456, + <<"nano_cpus">> => 1500000000, + <<"cpu_shares">> => 512 +} +``` + +输出: + +```json +{ + "HostConfig": { + "Memory": 536870912, + "MemoryReservation": 268435456, + "NanoCpus": 1500000000, + "CpuShares": 512 + } +} +``` + +值为 0 的资源字段会被省略。 + +### Ulimits + +输入: + +```erlang +[ + #{<<"name">> => <<"nofile">>, <<"soft">> => 1024, <<"hard">> => 2048} +] +``` + +输出: + +```json +{ + "HostConfig": { + "Ulimits": [ + { + "Name": "nofile", + "Soft": 1024, + "Hard": 2048 + } + ] + } +} +``` + +### Tmpfs + +输入: + +```erlang +#{ + <<"/tmp">> => <<>>, + <<"/run">> => <<"rw,size=64m">> +} +``` + +输出: + +```json +{ + "HostConfig": { + "Tmpfs": { + "/tmp": "", + "/run": "rw,size=64m" + } + } +} +``` + +### Sysctls + +输入: + +```erlang +#{<<"net.ipv4.ip_forward">> => <<"1">>} +``` + +输出: + +```json +{ + "HostConfig": { + "Sysctls": { + "net.ipv4.ip_forward": "1" + } + } +} +``` + +### ExtraHosts + +输入: + +```erlang +[<<"host.docker.internal:host-gateway">>] +``` + +输出: + +```json +{ + "HostConfig": { + "ExtraHosts": ["host.docker.internal:host-gateway"] + } +} +``` + +## 10. create.networking_config 到 Docker JSON + +输入: + +```erlang +#{ + <<"endpoints">> => [ + #{<<"name">> => <<"bridge">>}, + #{<<"name">> => <<"mynet">>} + ] +} +``` + +输出: + +```json +{ + "NetworkingConfig": { + "EndpointsConfig": { + "bridge": {}, + "mynet": {} + } + } +} +``` + +未传或 endpoints 为空时输出: + +```json +{ + "NetworkingConfig": {} +} +``` + +## 11. 完整转换示例 + +收到的 deploy command: + +```erlang +{<<"command">>, Ref, {<<"container">>, #{ + <<"action">> => <<"deploy">>, + <<"task_id">> => 1001, + <<"params">> => #{ + <<"container_name">> => <<"my_nginx">>, + <<"create">> => #{ + <<"config">> => #{ + <<"image">> => <<"docker.io/library/nginx:latest">>, + <<"cmd">> => [<<"nginx">>, <<"-g">>, <<"daemon off;">>], + <<"env">> => [<<"ENV=prod">>], + <<"volumes">> => [<<"/data">>], + <<"exposed_ports">> => [#{<<"container_port">> => 80, <<"protocol">> => <<"tcp">>}] + }, + <<"host_config">> => #{ + <<"binds">> => [<<"/host/data:/data">>], + <<"restart_policy">> => #{<<"name">> => <<"always">>, <<"maximum_retry_count">> => 0}, + <<"memory">> => 536870912, + <<"port_bindings">> => [ + #{ + <<"host_ip">> => <<>>, + <<"host_port">> => 8080, + <<"container_port">> => 80, + <<"protocol">> => <<"tcp">> + } + ] + }, + <<"networking_config">> => #{ + <<"endpoints">> => [#{<<"name">> => <<"bridge">>}] + } + } + } +}}} +``` + +生成的 Docker JSON 示意: + +```json +{ + "Image": "docker.io/library/nginx:latest", + "Cmd": ["nginx", "-g", "daemon off;"], + "Entrypoint": [], + "Env": ["CONTAINER_NAME=my_nginx", "ENV=prod"], + "Labels": {}, + "Volumes": { + "/usr/local/etc/service.conf": {}, + "/data": {} + }, + "User": "", + "WorkingDir": "", + "Hostname": "", + "ExposedPorts": { + "80/tcp": {} + }, + "NetworkingConfig": { + "EndpointsConfig": { + "bridge": {} + } + }, + "Healthcheck": {}, + "HostConfig": { + "Binds": [ + "/efka/root/my_nginx/service.conf:/usr/local/etc/service.conf", + "/host/data:/data" + ], + "RestartPolicy": { + "Name": "always" + }, + "Memory": 536870912, + "PortBindings": { + "80/tcp": [ + { + "HostIp": "", + "HostPort": "8080" + } + ] + } + } +} +``` + +## 12. 部署流程补充 + +deploy action 的执行流程: + +1. 根据 `container_name` 和 `docker.root_dir` 确保默认容器目录存在。 +2. 启动独立部署进程。 +3. 部署进程上报任务事件流。 +4. 规范化镜像名:如果镜像最后一段没有 tag,则补 `:latest`。 +5. 调用 Docker API 拉取镜像。 +6. 构造 Docker create JSON。 +7. 调用 `POST /containers/create?name={container_name}`。 +8. 创建空 `service.conf` 文件。 +9. 写入部署摘要日志并关闭任务事件流。 + +注意:当前 `ensure_container_absent/2` 只上报“开始创建容器”,不会删除已有同名容器;如果 Docker 返回名称冲突,部署会失败并上报“本地容器已经存在”。 diff --git a/docs/efka_iot_protocol.md b/docs/efka_iot_protocol.md new file mode 100644 index 0000000..fd39e7c --- /dev/null +++ b/docs/efka_iot_protocol.md @@ -0,0 +1,274 @@ +# EFKA 与 IOT 交互协议 + +本文档描述 `efka` 与 `iot` 之间的 TLS 长连接协议。当前协议由 Erlang term 直接序列化,发送端使用 `term_to_binary/1`,接收端使用 `binary_to_term(PacketBin, [safe])`。 + +协议帧只使用 safe external term:顶层 label、业务 label、map key 使用 binary;`Ref` 使用 `crypto:strong_rand_bytes(16)` 生成,是 16 字节 binary。网络协议里不发送 Erlang `reference()`,也不依赖动态创建 atom。 + +## 传输层 + +- `efka` 作为 TLS client 连接 `iot`。 +- `iot` 作为 TLS server 接收多个 `efka` 连接,一个连接对应一个 `ssl_channel` 进程。 +- socket 使用 `{packet, 4}`,每个 Erlang term binary 作为一个完整包发送。 +- `Ref` 使用 `crypto:strong_rand_bytes(16)` 生成,只在当前连接的 inflight 表内匹配。 + +## 顶层帧 + +协议顶层 tuple 用来表达交互语义: + +```erlang +{<<"request">>, Ref, Body} +{<<"response">>, Ref, Reply} +{<<"command">>, Ref, {Domain, Payload}} +{<<"command_response">>, Ref, {Domain, Reply}} +{<<"message">>, Body} +``` + +语义说明: + +| 帧 | 方向 | 语义 | +| --- | --- | --- | +| `{<<"request">>, Ref, Body}` | efka -> iot | efka 发起请求,需要 iot 回复 | +| `{<<"response">>, Ref, Reply}` | iot -> efka | iot 对 efka request 的回复 | +| `{<<"command">>, Ref, {Domain, Payload}}` | iot -> efka | iot 下发命令,需要 efka 回复 | +| `{<<"command_response">>, Ref, {Domain, Reply}}` | efka -> iot | efka 对 iot command 的回复 | +| `{<<"message">>, Body}` | 双向 | 异步消息,不要求回复 | + +`command` 和 `command_response` 的 `Domain` 表示业务域,目前支持: + +- `<<"container">>` + +## 鉴权请求 + +初始连接由 `efka` 发起鉴权 request。每条 TLS 连接只允许一次鉴权;`iot` 侧鉴权成功后会在 `ssl_channel` 标记该连接已鉴权,如果同一连接再次发送 `auth_request`,`iot` 会直接关闭连接。 + +```erlang +{<<"request">>, Ref, {<<"auth_request">>, #{ + <<"uuid">> => UUID, + <<"token">> => Token, + <<"timestamp">> => Timestamp +}}} +``` + +`iot` 回复: + +```erlang +{<<"response">>, Ref, {<<"auth_response">>, <<"ok">>}} +{<<"response">>, Ref, {<<"auth_response">>, {<<"error">>, {<<"failed">>, Reason}}}} +``` + +处理语义: + +- `<<"ok">>`:`efka` 进入 `activated` 状态。 +- `{<<"error">>, {<<"failed">>, Reason}}`:鉴权失败,`iot` 返回失败响应后关闭连接;`efka` 进入重连流程。 + +## 授权控制 + +`/host/activate` 只修改 `iot` 本地和持久化的 host 授权状态,不再向 `efka` 下发 auth command。`efka` 可以继续保持连接并发送数据,是否处理这些数据由 `iot_host` 当前状态决定。 + +因此当前协议没有 `{<<"command">>, Ref, {<<"auth">>, ...}}` 和 `{<<"command_response">>, Ref, {<<"auth">>, ...}}`。授权关闭时,`iot_host` 保持 channel 在线,但不处理上报数据;授权重新打开后,已在线的 channel 可以继续使用。 + +## 容器管理命令 + +`iot` 对 `efka` 的容器管理使用 command 语义: + +```erlang +{<<"command">>, Ref, {<<"container">>, CommandMap}} +``` + +`efka` 回复: + +```erlang +{<<"command_response">>, Ref, {<<"container">>, Reply}} +``` + +`Reply` 取值: + +```erlang +<<"ok">> +{<<"ok">>, Result} +{<<"error">>, Reason} +``` + +`CommandMap` 使用 binary key 和 binary action;`efka` 接收后直接按 binary key/action 匹配,Docker 参数链路继续使用 binary-key map,不再转换成 atom-key map。 + +### list + +```erlang +#{<<"action">> => <<"list">>} +``` + +返回当前 `efka` 主机上的容器列表。 + +### deploy + +```erlang +#{ + <<"action">> => <<"deploy">>, + <<"task_id">> => TaskId, + <<"params">> => Params +} +``` + +触发容器部署。部署过程中的流式日志不通过该 command response 返回,而是通过 `message` 的 `task_event` 上报。 + +### start + +```erlang +#{ + <<"action">> => <<"start">>, + <<"target">> => Target +} +``` + +### stop + +```erlang +#{ + <<"action">> => <<"stop">>, + <<"target">> => Target, + <<"timeout_seconds">> => TimeoutSeconds +} +``` + +### kill + +```erlang +#{ + <<"action">> => <<"kill">>, + <<"target">> => Target, + <<"signal">> => Signal +} +``` + +### remove + +```erlang +#{ + <<"action">> => <<"remove">>, + <<"target">> => Target, + <<"force">> => Force, + <<"remove_volumes">> => RemoveVolumes +} +``` + +### config + +```erlang +#{ + <<"action">> => <<"config">>, + <<"target">> => Target, + <<"config">> => Config +} +``` + +更新容器配置文件。 + +### Target + +容器目标使用 map 表示: + +```erlang +#{ + <<"name">> => ContainerName, + <<"id">> => ContainerId +} +``` + +`name` 和 `id` 至少一个非空;优先使用 `name`,`name` 为空时使用 `id`。 + +## 异步消息 + +`message` 不带 `Ref`,不要求对端回复。 + +### efka -> iot: data + +```erlang +{<<"message">>, {<<"data">>, #{ + <<"route_key">> => RouteKey, + <<"metric">> => Metric +}}} +``` + +用于 `efka` 上报业务指标数据。 + +### efka -> iot: task_event + +```erlang +{<<"message">>, {<<"task_event">>, #{ + <<"task_id">> => TaskId, + <<"type">> => Type, + <<"stream">> => Stream +}}} +``` + +任务事件流关闭时: + +```erlang +{<<"message">>, {<<"task_event">>, #{ + <<"task_id">> => TaskId, + <<"type">> => <<"close">>, + <<"stream">> => Reason +}}} +``` + +`task_event` 本身不携带 `uuid`。`iot` 在接收该消息时使用当前已鉴权 `ssl_channel` 绑定的 host UUID,把事件路由到内部任务进程 `{UUID, TaskId}`。HTTP 页面通过 SSE 订阅时也必须使用同一组参数: + +```http +GET /event_stream?uuid=&task_id= +``` + +`iot` 会为每个 `{UUID, TaskId}` 维护一个独立的任务进程,用于缓存最近的部署日志、支持多个 SSE listener,并在收到 close 事件后结束事件流。 + +### efka -> iot: ping + +```erlang +{<<"message">>, <<"ping">>} +``` + +用于 TLS 长连接的应用层保活。`efka` 在鉴权成功后周期发送,当前发送间隔为 30 秒。 + +`iot` 收到后回复: + +```erlang +{<<"message">>, <<"pong">>} +``` + +`ping/pong` 只表示 TLS 连接仍可读写,不参与 host online/offline 判定。host 上下线仍由 UDP 心跳和 `iot_host` 本地连接状态共同维护。 + +### iot -> efka: pub + +```erlang +{<<"message">>, {<<"pub">>, #{ + <<"topic">> => Topic, + <<"qos">> => Qos, + <<"content">> => Content +}}} +``` + +用于 `iot` 向 `efka` 本地订阅系统发布 topic 消息。 + +## 状态与超时 + +- `efka` 鉴权超时时间:5 秒。 +- `iot` command inflight 超时时间:60 秒。 +- `iot` SSL channel 空闲超时时间:120 秒。120 秒内没有收到任何 TLS 包,包括 `ping`、业务 `message`、`request`、`command_response`,`iot` 会主动关闭该连接。 +- `iot` 管理多个 `efka` 时,每个连接有独立 `ssl_channel` 和独立 inflight 表。 +- command 超时后,`iot` 删除 inflight 记录;之后如果迟到的 `command_response` 到达,会被视为未预期响应。 + +## UDP 心跳 + +`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。TLS control channel 和 UDP 心跳共用 `iot_server.host`,分别使用 `tls_port` 和 `udp_port`。UDP 心跳包使用 HMAC-SHA256 校验,HMAC key 为 `SHA256(auth.token)`。 + +详细格式见 [heartbeat.md](heartbeat.md)。 + +## 兼容性 + +当前协议不兼容旧 tuple: + +- 旧容器管理:`{request, Ref, {container_request, ...}}` +- 旧容器回复:`{response, Ref, {container_response, ...}}` +- 旧授权控制:`{message, {auth_control, Command}}` +- 已移除的 auth command:`{command, Ref, {auth, activate | deactivate}}` +- 旧 Ref:Erlang `reference()`,例如 `make_ref()` 生成的值。 + +如果需要滚动升级,应先增加临时兼容分支或引入协议版本协商。 diff --git a/docs/heartbeat.md b/docs/heartbeat.md new file mode 100644 index 0000000..4efee8d --- /dev/null +++ b/docs/heartbeat.md @@ -0,0 +1,52 @@ +# UDP 心跳 + +`efka` 通过独立的 `efka_iot_heartbeat` 进程向 `iot` 发送 UDP 心跳。该心跳只表示主机存活,不依赖 TLS control channel 是否在线。 + +## 配置 + +TLS 和 UDP 共用同一个 `iot_server.host`: + +```erlang +{iot_server, [ + {host, "localhost"}, + {tls_port, 443}, + {udp_port, 18080} +]}, + +{heartbeat, [ + {interval, 5000} +]}, + +{auth, [ + {uuid, "qbxmjyzrkpntfgswaevodhluicqzxplkm"}, + {token, "zpxlkvmqwnbghytrujsdieofazxcvbnm"} +]} +``` + +- `tls_port` 用于 `efka_iot_client` 建立 TLS 连接。 +- `udp_port` 用于 `efka_iot_heartbeat` 发送 UDP 心跳。 +- `heartbeat.interval` 是发送间隔,单位毫秒,默认 5000。 +- UDP 心跳和 TLS 鉴权复用 `auth.uuid` 和 `auth.token`。 + +## 包格式 + +```erlang +<< + Version:8, + UuidLen:16, + UUID:UuidLen/binary, + Timestamp:64/unsigned-big, + Nonce:16/binary, + Mac:32/binary +>> +``` + +`Mac` 使用 HMAC-SHA256: + +```erlang +HeartbeatSecret = crypto:hash(sha256, Token), +Payload = <>, +Mac = crypto:mac(hmac, sha256, HeartbeatSecret, Payload) +``` + +其中 `Token` 是 `auth.token`。`iot` 侧注册 efka client 时保存同样的 `SHA256(Token)` 派生值,并用它校验心跳。 diff --git a/docs/service_websocket_protocol.md b/docs/service_websocket_protocol.md new file mode 100644 index 0000000..45119c3 --- /dev/null +++ b/docs/service_websocket_protocol.md @@ -0,0 +1,439 @@ +# EFKA 与 Service WebSocket 交互协议 + +本文档描述部署在 `efka` 主机上的 service 与 `efka` 之间的 WebSocket 通讯逻辑。当前接入实现位于: + +- [efka_service_channel.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service_channel.erl) +- [service.proto](/usr/local/code/cloudkit/efka/apps/efka/proto/service.proto) +- [efka_subscription.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_subscription.erl) +- [efka_service.erl](/usr/local/code/cloudkit/efka/apps/efka/src/service/efka_service.erl) + +## 1. 连接入口 + +`efka` 启动时会创建 Cowboy WebSocket server,监听配置来自 `efka.config` 的 `websocket_server`: + +```erlang +{websocket_server, [ + {port, 18080}, + {acceptors, 10}, + {max_connections, 1024}, + {backlog, 1024} +]} +``` + +WebSocket 路由固定为: + +```http +GET /ws +``` + +连接建立后,每个 WebSocket 连接对应一个 `efka_service_channel` 进程。 + +当前协议没有独立的认证字段。service 必须先发送 `register` request 完成注册;未注册前只能处理 `register`,其它 request 会返回 `invalid request`,cast 会被忽略。 + +Cowboy WebSocket ping 帧由 `efka_service_channel` 直接回复 pong: + +```erlang +websocket_handle(ping, State) -> {reply, pong, State} +``` + +## 2. 二进制帧格式 + +service 与 `efka` 之间使用 WebSocket binary frame。每个业务 frame 的第 1 个字节是自定义帧类型,剩余字节是 protobuf payload。 + +```text + +``` + +帧类型: + +| frame_type | 十六进制 | 方向 | 语义 | protobuf 类型 | +| --- | --- | --- | --- | --- | +| `REQUEST` | `0x01` | service -> efka | 请求,需要 efka 回复 | `ServiceRequest` | +| `REPLY` | `0x02` | efka -> service | request 的响应 | `ServiceReply` | +| `CAST` | `0x03` | 双向 | 单向消息,不要求回复 | `ServiceCast` | + +`efka_service_channel` 当前只处理 service 发来的 `REQUEST` 和 `CAST`。收到未知格式时只记录日志,不主动发送错误响应。 + +## 3. Protobuf 定义 + +协议定义来自 `apps/efka/proto/service.proto`: + +```protobuf +syntax = "proto3"; + +message ServiceRequest { + uint32 packet_id = 1; + + message Register { + string service_id = 1; + } + + message Subscribe { + string topic = 1; + } + + oneof request { + Register register = 10; + Subscribe subscribe = 11; + } +} + +message ServiceReply { + uint32 packet_id = 1; + + message Error { + int32 code = 1; + string message = 2; + } + + oneof reply { + bytes result = 10; + Error error = 11; + } +} + +message ServiceCast { + message MetricData { + bytes route_key = 1; + bytes metric = 2; + } + + message TopicEvent { + string topic = 1; + bytes content = 2; + } + + oneof body { + TopicEvent topic_event = 10; + MetricData metric_data = 11; + } +} +``` + +注意: + +- `ServiceRequest.packet_id` 由 service 生成,用于匹配 request 和 reply。 +- `ServiceReply.packet_id` 必须等于原 request 的 `packet_id`。 +- `ServiceReply.result` 当前成功值一般为 `<<"ok">>`。 +- `ServiceReply.error.code` 当前失败时统一使用 `-1`。 +- `MetricData.route_key` 和 `MetricData.metric` 是 bytes。 +- `TopicEvent.topic` 是 string,`TopicEvent.content` 是 bytes。 + +## 4. request/reply 语义 + +### 4.1 register + +service 建立 WebSocket 后应先发送注册请求: + +```protobuf +ServiceRequest { + packet_id: 1 + register { + service_id: "service-a" + } +} +``` + +WebSocket binary frame: + +```text +0x01 ++ protobuf(ServiceRequest) +``` + +`efka` 处理逻辑: + +1. `efka_service_channel` 解码 `ServiceRequest`。 +2. 调用 `efka_service_sup:start_service(ServiceId)` 启动或复用 service 进程。 +3. 调用 `efka_service:attach_channel(ServicePid, ChannelPid)` 绑定当前 WebSocket channel。 +4. 通过 `efka_service_model:insert/1` 写入 service 记录: + +```erlang +#service{ + service_id = ServiceId, + container_name = <<>>, + status = ?SERVICE_RUNNING, + meta_data = #{}, + create_ts = efka_util:timestamp(), + update_ts = efka_util:timestamp() +} +``` + +如果该 `service_id` 是新记录,`status` 会写入 `?SERVICE_RUNNING`。如果该 `service_id` 已存在,当前 `efka_service_model:insert/1` 只更新 `meta_data`、`container_name` 和 `update_ts`,不会覆盖已有 `status`。 + +5. `efka_service_channel` 状态更新为: + +```erlang +#state{ + service_id = ServiceId, + service_pid = ServicePid, + is_registered = true +} +``` + +成功响应: + +```protobuf +ServiceReply { + packet_id: 1 + result: "ok" +} +``` + +WebSocket binary frame: + +```text +0x02 ++ protobuf(ServiceReply) +``` + +失败响应: + +```protobuf +ServiceReply { + packet_id: 1 + error { + code: -1 + message: "attach channel failed" + } +} +``` + +常见失败场景: + +- 同一个 `service_id` 已有存活 channel,`efka_service:attach_channel/2` 返回 `{error, <<"channel exists">>}`。 +- 当前实现对外统一返回 `"attach channel failed"`,详细原因只写日志。 + +### 4.2 subscribe + +注册成功后,service 可以订阅 topic: + +```protobuf +ServiceRequest { + packet_id: 2 + subscribe { + topic: "device/*/event" + } +} +``` + +`efka` 处理逻辑: + +1. 要求当前 channel 已注册,即 `is_registered = true`。 +2. 调用 `efka_subscription:subscribe(Topic, ChannelPid)`。 +3. 订阅成功后,把 topic 加入 `efka_service_channel` 的 `subscribed_topics` 集合。 + +成功响应: + +```protobuf +ServiceReply { + packet_id: 2 + result: "ok" +} +``` + +失败响应: + +```protobuf +ServiceReply { + packet_id: 2 + error { + code: -1 + message: "invalid topic name" + } +} +``` + +未注册时发送 subscribe 会走通用错误: + +```protobuf +ServiceReply { + packet_id: 2 + error { + code: -1 + message: "invalid request" + } +} +``` + +## 5. cast 语义 + +### 5.1 service -> efka: metric_data + +service 通过 `CAST` 上报指标数据: + +```protobuf +ServiceCast { + metric_data { + route_key: "device/a/temp" + metric: "{\"value\":23.5}" + } +} +``` + +WebSocket binary frame: + +```text +0x03 ++ protobuf(ServiceCast) +``` + +`efka` 处理逻辑: + +1. `efka_service_channel` 解码 `ServiceCast`。 +2. 要求当前 channel 已注册。 +3. 调用: + +```erlang +efka_service:metric_data(ServicePid, RouteKey, Metric) +``` + +4. `efka_service` 再调用: + +```erlang +efka_iot_client:metric_data(RouteKey, Metric) +``` + +5. `efka_iot_client` 通过 efka 与 iot 的 TLS 长连接把数据上报给 iot: + +```erlang +{<<"message">>, {<<"data">>, #{ + <<"route_key">> => RouteKey, + <<"metric">> => Metric +}}} +``` + +`metric_data` 是单向消息,service 不会收到 reply。未注册或未知 cast body 会被忽略。 + +### 5.2 efka -> service: topic_event + +当 `iot` 通过 efka 与 iot 的 TLS channel 下发 pub 消息到 `efka` 时,`efka_iot_client` 会调用: + +```erlang +efka_subscription:publish(Topic, Qos, Content) +``` + +`efka_subscription` 根据 topic 匹配已订阅的 `efka_service_channel`。匹配成功后向 channel 进程发送: + +```erlang +{topic_broadcast, Topic, Content} +``` + +`efka_service_channel` 再推送 WebSocket `CAST` 给 service: + +```protobuf +ServiceCast { + topic_event { + topic: "device/a/event" + content: "payload bytes" + } +} +``` + +WebSocket binary frame: + +```text +0x03 ++ protobuf(ServiceCast) +``` + +`topic_event` 是单向消息,不要求 service 回复。 + +## 6. topic 匹配规则 + +`efka_subscription` 当前使用 `/` 拆分 topic: + +```erlang +binary:split(Topic, <<$/>>, [global]) +``` + +支持两种通配符: + +| 通配符 | 语义 | +| --- | --- | +| `*` | 单级匹配,只匹配一个 segment | +| `+` | 多级匹配,只能出现在最后一段,且至少匹配一个剩余 segment | + +示例: + +| 订阅 topic | 发布 topic | 是否匹配 | +| --- | --- | --- | +| `device/a/temp` | `device/a/temp` | 是 | +| `device/*/temp` | `device/a/temp` | 是 | +| `device/*/temp` | `device/a/b/temp` | 否 | +| `device/+` | `device/a` | 是 | +| `device/+` | `device/a/temp` | 是 | +| `device/+` | `device` | 否 | + +同一个 `efka_service_channel` 订阅多个 topic 时,同一次 publish 最多投递一次。 + +## 7. QoS 与遗留消息 + +`efka_subscription:publish(Topic, Qos, Content)` 中的 `Qos` 影响没有订阅者时的行为: + +- `Qos = 0`:没有匹配订阅者时直接丢弃。 +- `Qos /= 0`:没有匹配订阅者时暂存到 `remaining_messages`。 + +当后续有 service 订阅能匹配这些 topic 时,`efka_subscription` 会把对应遗留消息推送给该 service,并从 `remaining_messages` 移除。 + +当前 `remaining_messages` 存在内存中,不持久化;`efka` 重启后会丢失。 + +## 8. 生命周期与清理 + +### WebSocket channel 关闭 + +`efka_service_channel:terminate/3` 会执行: + +1. 调用 `efka_subscription:unsubscribe_all(ChannelPid)` 清理该 channel 的所有订阅。 +2. 如果 channel 已完成 register,则调用: + +```erlang +efka_service_model:change_status(ServiceId, ?SERVICE_STOPPED) +``` + +也就是把 service 状态更新为停止。 + +### service 进程退出 + +`efka_service_channel` monitor 绑定的 `efka_service` 进程。如果 service 进程退出,channel 会停止: + +```erlang +websocket_info({'DOWN', _Ref, process, ServicePid, Reason}, State) -> + {stop, State#state{service_pid = undefined}} +``` + +`efka_service` 同时也 monitor channel。channel 退出时,`efka_service` 会清空自身的 `channel_pid`。 + +## 9. 完整交互流程 + +典型流程: + +```text +service efka/efka_service_channel efka_service efka_subscription + | | | | + |-- WebSocket connect /ws ------------>| | | + | | | | + |-- REQUEST register ----------------->| | | + | |-- start_service(service_id) ->| | + | |-- attach_channel ----------->| | + |<------------- REPLY result="ok" -----| | | + | | | | + |-- REQUEST subscribe ---------------->| | | + | |---------------- subscribe ------------------------------>| + |<------------- REPLY result="ok" -----| | | + | | | | + |-- CAST metric_data ----------------->| | | + | |-- metric_data -------------->| | + | | |-- efka_iot_client:metric_data -> iot + | | | | + |<------------- CAST topic_event ------|<--------------- topic_broadcast -----------------------| + | | | | + |-- WebSocket close ------------------>| | | + | |-- unsubscribe_all -------------------------------------->| + | |-- change_status(service_id, stopped) +``` + +## 10. 当前限制 + +- 没有认证字段,service 只通过 `service_id` 注册。 +- `register` 可重复连接同一个 `service_id`,但同一时间只允许一个活跃 channel 绑定到 `efka_service`。 +- request 只支持 `register` 和 `subscribe`。 +- cast 只支持 `metric_data` 和 `topic_event`。 +- `subscribed_topics` 目前只在 `efka_service_channel` 内记录,关闭时实际清理由 `efka_subscription:unsubscribe_all/1` 完成。 +- `efka_subscription` 当前基于列表扫描匹配,订阅规模很大时需要后续优化为索引结构。 diff --git a/docs/todo.md b/docs/todo.md new file mode 100644 index 0000000..acdd5de --- /dev/null +++ b/docs/todo.md @@ -0,0 +1,50 @@ +# TODO + +## 协议层 + +### 高优先级 + +- 将 `message.proto` 收敛为 `efka` 与 `iot` 共用的一份单一事实来源,避免协议定义漂移和双端重复维护。 + +### 中优先级 + +- 增强 protobuf 映射和 Docker JSON 生成相关的自动化契约测试。 + +## Docker 层 + +### 高优先级 + +- 将 `docker_commands` 拆分为更清晰的层次:Docker HTTP 客户端、Docker JSON 构造层、以及 efka 本地补丁逻辑。 +- 明确 `docker_task_reporter` 的投递语义;当前只有内存缓冲,还缺少持久化能力和队列上限控制。 + +### 中优先级 + +- 丰富 `docker_deploy_manager` 的任务跟踪状态,把部署元数据、耗时、阶段、失败原因等信息纳入统一管理,避免后续再回头重构。 +- 统一 `docker_container_service` 的返回结构,避免调用方分支处理 `ok | {ok, binary()} | {error, binary()}` 这种混合形式。 +- 消除 `docker_commands` 中重复的 Docker HTTP 响应解析逻辑。 + +### 低优先级 + +- 对于像 `docker_events.erl` 这样当前未启用的模块,要么清理掉,要么补充说明其保留原因。 + +## 订阅与通道层 + +### 高优先级 + +- 将 `efka_subscription` 当前基于列表扫描的匹配方式替换为更可扩展的索引结构,例如 trie 或基于 ETS 的索引。 + +### 中优先级 + +- 继续收敛 `efka_iot_client` 的职责,将传输层状态管理与 request/cast 协议处理进一步拆开。 +- 评估并落实 `efka_service_channel` 中 `subscribed_topics` 集合的用途;如果只是被动保存状态,则应简化。 + +### 低优先级 + +- 如果 `efka_iot_client` 的缓存指标量继续增长,可以为缓存刷出增加批量发送或节流机制。 + +## 基础设施层 + +### 低优先级 + +- 如果上传相关能力已经不再属于当前运行时,移除或补充说明残留的启动逻辑,例如 `ensure_upload_dir/0`。 +- 评估是否将 `efka_logger` 与 OTP `logger` 统一,避免长期维护两套日志链路。 diff --git a/message_pb.proto b/message_pb.proto index 0e8112a..8c7aa67 100644 --- a/message_pb.proto +++ b/message_pb.proto @@ -19,23 +19,34 @@ message AuthReply { // service_id主动订阅消息, 基于广播通讯 message Pub { string topic = 1; - string content = 2; + bytes content = 2; +} + +message Command { + string command_type = 1; + bytes command = 2; } ///// 服务器主动推送的消息 -message AsyncCallReply { - // 0: 表示失败,1: 成功 - uint32 code = 1; - string result = 2; - string message = 3; +// 部署逻辑 +message RPCDeploy { + uint32 task_id = 1; + // json + string config = 2; } -// 部署逻辑 -message Deploy { - uint32 task_id = 1; - string service_id = 2; - string tar_url = 3; +message RPCStartContainer { + string container_name = 1; +} + +message RPCStopContainer { + string container_name = 1; +} + +message RPCConfigContainer { + string container_name = 1; + bytes config = 2; } // 获取task的logs @@ -43,18 +54,11 @@ message FetchTaskLog { uint32 task_id = 1; } -// 需要响应, 云端主动发起的调用; 提供给用户 -message Invoke { - string service_id = 1; - string payload = 2; - uint32 timeout = 3; -} - // 参数配置 -message PushServiceConfig { - string service_id = 1; - string config_json = 2; - uint32 timeout = 3; +message ContainerConfig { + string container_name = 1; + // 任意的数据格式 + bytes config = 2; } /////// EFKA主动上报的消息类型 @@ -63,8 +67,15 @@ message PushServiceConfig { message Data { string service_id = 1; string device_uuid = 2; + string route_key = 3; // measurement[,tag_key=tag_value...] field_key=field_value[,field_key2=field_value2...] [timestamp] - string metric = 3; + bytes metric = 4; +} + +message Event { + string service_id = 1; + uint32 event_type = 2; + string params = 3; } //#{<<"adcode">> => 0,<<"boot_time">> => 18256077,<<"city">> => <<>>, @@ -97,24 +108,4 @@ message Ping { repeated int32 memory = 12; // 接口信息的定义: 每个接口的信息, 采用json格式传输,没有办法提前定义 string interfaces = 13; -} - -// Inform消息 -message ServiceInform { - string service_id = 1; - string props = 2; - uint32 status = 3; - uint32 timestamp = 4; -} - -message Event { - string service_id = 1; - uint32 event_type = 2; - string params = 3; -} - -// 告警信息 -message Alarm { - string service_id = 1; - string params = 2; } \ No newline at end of file diff --git a/rebar.config b/rebar.config index b91b703..3400147 100644 --- a/rebar.config +++ b/rebar.config @@ -1,11 +1,15 @@ {erl_opts, [debug_info]}. + +{project_app_dirs, ["apps/*"]}. + +{plugins, [ + {rebar3_gpb_plugin, ".*", {git, "https://github.com/lrascao/rebar3_gpb_plugin.git", {tag, "2.23.8"}}} +]}. + {deps, [ {sync, ".*", {git, "https://github.com/rustyio/sync.git", {branch, "master"}}}, - {jiffy, ".*", {git, "https://github.com/davisp/jiffy.git", {tag, "1.1.2"}}}, - {gpb, ".*", {git, "https://github.com/tomas-abrahamsson/gpb.git", {tag, "4.20.0"}}}, - {jiffy, ".*", {git, "https://github.com/davisp/jiffy.git", {tag, "1.1.1"}}}, - {parse_trans, ".*", {git, "https://github.com/uwiger/parse_trans", {tag, "3.0.0"}}}, - {lager, ".*", {git,"https://github.com/erlang-lager/lager.git", {tag, "3.9.2"}}} + {cowboy, ".*", {git, "https://github.com/ninenines/cowboy.git", {tag, "2.10.0"}}}, + {gun, ".*", {git, "https://github.com/ninenines/gun.git", {tag, "2.2.0"}}} ]}. {relx, [{release, {efka, "0.1.0"}, @@ -38,11 +42,4 @@ ] }]}]}. -{project_plugins, [ - %% 或从 Git 仓库拉取最新版本 - {pc, {git, "https://github.com/blt/port_compiler.git", {tag, "v1.15.0"}}} -]}. - -{erl_opts, [{parse_transform,lager_transform}]}. - -{rebar_packages_cdn, "https://hexpm.upyun.com"}. \ No newline at end of file +{rebar_packages_cdn, "https://hexpm.upyun.com"}.