[feat][evaluation]eval0630#581
Open
tpfz wants to merge 139 commits into
Open
Conversation
…for item-centric experiment refactoring - Add expt_item_ref table (new flat item-binding table with item_config) - Add item_version_id to expt_item_result/run_log, expt_turn_result/run_log, eval_target_record, evaluator_record - Add source_type/inline_key/alias/target_record_id to evaluator_record (Builtin/Alias/Inline unification) - Add source_type/inline_key/alias to expt_turn_evaluator_result_ref; rebuild 7-col unique key - Add eval_set_source_type to experiment (new/legacy path discriminator) - Add alias/filter/binding_config/eval_set_id to expt_evaluator_ref (query-only snapshot) - Update generate.go: add expt_item_ref to experiment tables; add evaluator_version/evaluator_record to evaluator tables - Regenerate all affected gorm_gen model/query files Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…experiment - domain/expt.thrift: add ExptEvalSetSourceType enum, EvalSetConfig/ExptTargetConf/ ExptEvaluatorConf/ExptFilter/ExptFilterField/ExptEvalSetDetail structs - domain/expt.thrift: add fields 110-114 to Experiment DTO (eval_set_source_type, eval_set_configs, eval_set_details, evaluators_concur_num, total_item_count) - coze.loop.evaluation.expt.thrift: add eval_set_configs field 70 to CreateExperimentRequest, field 75 to SubmitExperimentRequest - regenerate kitex_gen for evaluation domain Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… EvaluatorResults - expt.go: add ExptEvalSetSourceType enum, ExptItemRef/ExptItemConfig structs, EvalSetConfig/ExptTargetConf/ExptEvaluatorConf/ExptItemFilter domain types; add EvalSetSourceType to Experiment, EvalSetConfigs to EvaluationConfiguration - expt_result.go: add ItemVersionID to ExptItemResult/ExptItemResultRunLog/ ExptTurnResult/ExptTurnResultRunLog; upgrade EvaluatorResults to dual-array format (Registered+Inline) with legacy EvalVerIDToResID backward compat; add SourceType/InlineKey/Alias to ExptTurnEvaluatorResultRef; add Alias/Filter/BindingConfig/EvalSetID to ExptEvaluatorRef - evaluator_record.go: add ItemVersionID, SourceType, InlineKey, Alias, TargetRecordID fields; add EvaluatorRecordSourceType enum; add EvaluatorRunStatusSkipped constant - target_record.go: add ItemVersionID field Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…etItemCount entity - Add IExptItemRefRepo interface with BatchCreate, ListByExptID, GetByExptIDAndItemID, MGetByExptIDAndItemIDs, CountByEvalSetGrouped methods - Add ExptEvalSetItemCount struct for eval_set grouped item counts - Update go:generate directive to include IExptItemRefRepo Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ields in converters - mysql/expt_item_ref.go: IExptItemRefDAO interface + implementation (BatchCreateNX, ListByExptID, GetByExptIDAndItemID, MGetByExptIDAndItemIDs, CountByEvalSetGrouped) - experiment/expt_item_ref.go: IExptItemRefRepo implementation wiring DAO + converter - convert/expt_item_ref.go: ExptItemRef DO/PO converter with item_config JSON marshaling - convert/expt_result.go: propagate ItemVersionID in ExptItemResult/ExptTurnResult converters - convert/expt_item_result_run_log.go: propagate ItemVersionID - convert/expt_run_log.go: propagate ItemVersionID in ExptTurnResultRunLog converter - convert/expt_turn_evaluator_result_ref.go: add SourceType/InlineKey/Alias fields - convert/expt_evaluator_ref.go: add EvalSetID/Alias/Filter/BindingConfig fields - evaluator/convertor/evaluator_record.go: add all new fields (ItemVersionID, SourceType, InlineKey, Alias_, TargetRecordID) - target/convertor/eval_target_record.go: add ItemVersionID - domain/repo/expt.go: add IExptItemRefRepo interface Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… convertor - entity/param.go: add EvalSetConfigs to CreateExptParam - entity/expt.go: update ToEvaluatorRefDO to generate alias/filter/binding_config for MultiSetConfig path; add EvalSetSourceType to Experiment struct - service/expt_manage_impl.go: detect EvalSetConfigs in CreateExpt; set EvalSetSourceType=MultiSetConfig or SingleSet accordingly; serialize EvalSetConfigs into eval_conf - infra/convert/expt.go: propagate EvalSetSourceType in DO2PO/PO2DO - application/convertor/experiment/expt.go: ConvertCreateReq handles eval_set_configs field 70 → domain EvalSetConfigs; ToExptDTO fills fields 110-114 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…atorResults upgrade
ExptStart (Task 6):
- ExptSubmitExec: add exptItemRefRepo field (variadic backward-compat constructor)
- ExptStart: detect EvalSetSourceType==MultiSetConfig → call exptStartMultiSet
- exptStartMultiSet: paginate items per set, build ExptItemRef with item_config,
batch-write expt_item_ref then expt_item_result/expt_turn_result
- buildItemConfigFromSetConf: materialize per-set evaluator/target conf into item_config
storeTurnRunResult / RecordItemRunLogs (Task 7):
- storeTurnRunResult: write new EvaluatorResults{Registered:[], Alias:''} format
- NewTurnEvaluatorResultRefs: handle both old map format and new Registered+Inline arrays;
set SourceType=Builtin for registered, SourceType=Inline for inline
- terminateZombieEvaluatorRecords: extract record IDs from both old/new format
- RecordItemRunLogs: extract evaluator IDs from both formats for weight calculation
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- expt_result_impl_test.go: TestNewTurnEvaluatorResultRefs_NewFormat covers nil/old-map/Registered/Inline/mixed cases with SourceType assertion - expt_result_eval_test.go: TestEvaluatorResultsSerializeCompat covers new-format serialize, old-format JSON backward-compat, new-format deserialize - expt_convertor_test.go: TestConvertCreateReq_OldPath/EvalSetConfigs, TestConvertEvalSetConfigsDTOToDO - application convertor new path - expt_run_scheduler_mode_multiset_test.go: TestBuildItemConfigFromSetConf covers empty/single-evaluator/with-target-conf cases All 11 new test cases PASS Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…s collision Replace sync.Map(int64 key) with mutex+slice (evalRecordCollector) in callEvaluators/asyncCallEvaluator so alias multi-instances of the same evaluatorVersionID are stored as separate records instead of overwriting each other. ExptTurnRunResult.EvaluatorResults changes from map[int64]*EvaluatorRecord to []*EvaluatorRecord; GetEvaluatorRecord uses linear search by versionID for backward compatibility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
新实验类型 (MultiSetConfig) 聚合时跳过 EvaluatorScore/WeightedScore 维度
(多评测集 + alias 多实例下"实验级单一平均分"语义不成立), 保留 Target
latency/tokens + Annotation 维度。附加 Skipped record 过滤兜底和 field_key
兼容解析工具。
- domain/entity/aggr_field_key.go: 新增 ParseEvaluatorScoreFieldKey 工具,
兼容纯数字 (当前 DB 行) 和 verID:alias (未来扩展)
- expt_result_aggr_impl.go:
* CreateExptAggrResult 入口按 EvalSetSourceType 分流, 提取
computeEvaluatorAggrGroup 为独立方法
* createWeightedScoreAggrResult 入口 guard MultiSetConfig 返回 nil
* UpdateExptAggrResult 入口 guard MultiSetConfig 早返回
* 两处 recordMap 构建加 Status=Skipped 过滤
* 3 处 ParseInt(FieldKey) 切换为 ParseEvaluatorScoreFieldKey
- filter.go: ConvertFilter ParseInt 切兼容解析
- 新增 11 个单测 (7 个工具 + 4 个分流), 既有聚合测试 0 回归
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
新 item-centric 路径 (MultiSetConfig) 下 ConvertCreateReq 只设 EvalSetConfigs， 未构建 EvalConf.ConnectorConf，而 CheckRun->CheckConnector->checkEvaluatorsConnector 仍按老连接器结构做同步字段映射校验，导致 SubmitExperiment 报 'invalid evaluator connector' (cause: nil EvaluatorConf)。 方案A: 新增 buildExptConfFromEvalSetConfigs，从 eval_set_configs[].evaluator_confs /target_confs 的字段映射展平派生老连接器 (EvaluatorsConf + TargetConf)， 按 (evaluator_version_id, alias) 去重，使两侧由同一份输入派生、天然一致。 EvalSetConfigs 仍是落库与调度的权威源。 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
新实验类型 (MultiSetConfig) 执行 evaluator 时按 expt_item_ref.item_config
的 EvaluatorConfs 行级配置做 filter 判断, filter 不命中则跳过该 evaluator
(不实际调用 evaluator service)。老实验类型走原 expt 级 EvaluatorsConf 路径,
零行为变化。
- domain/entity/expt_run.go:
* ExptItemEvalCtx 加 ItemConfig 字段
* ExptTurnRunResult 加 GetEvaluatorRecordByVerAlias(versionID, alias) 方法,
为后续 alias 多实例独立定位预留 (老 GetEvaluatorRecord 保留)
- domain/service/expt_run_item_event_impl.go:
* ExptItemEventEvalServiceImpl 加 exptItemRefRepo 字段; NewExptRecordEvalService
构造函数加 IExptItemRefRepo 参数
* BuildExptRecordEvalCtx 按 EvalSetSourceType 分流: MultiSetConfig 调
GetByExptIDAndItemID 拿 item_config 注入 ctx; 读不到降级 nil (走老路径)
- domain/service/expt_item_filter_match.go (新增):
* ShouldRunByFilter 综合 FilterMode (None/Include/Exclude) + matcher 决策
* MatchExptItemFilter 支持 QueryAndOr (AND/OR) + 多 FilterField
* matchByQueryType 支持 equal/in/not_equal/not_in/contains/not_contains;
未识别 QueryType 默认放行 (不阻断执行)
* shouldRunEvaluatorByItemConfig 桥接器: 找到 versionID 对应的首个 conf
做 filter 判定
- domain/service/expt_run_item_turn_impl.go:
* CallEvaluators 入口加 ItemConfig 守卫: filter 不命中 → continue 跳过
该 evaluator, 不进 pendingEvaluatorVersionIDs
- wire: experiment/wire.go 加 NewExptItemRefRepo, mysql/wire.go 加
NewExptItemRefDAO, wire_gen.go 自动重新生成 (含 iExptItemRefRepo 变量)
- 新增 18 个单测 (12 filter matcher + 6 桥接器 + 4 双键查找), 既有测试 0 回归
tech debt (后续 PR):
- alias 多实例独立执行 + Skipped 占位 record 持久化 — 依赖 evaluatorService
.RunEvaluator API 扩展 Alias/SourceType 字段; 当前 MVP 只是不实际调用
evaluator (filter 行为正确), 但 GUI 看不到 Skipped 标记
- 同 versionID 多 alias 实例只取首个 conf 判定; 等 alias 真正落地后切换
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PR2 只在 BuildExptRecordEvalCtx (record eval 路径) 注入了 IExptItemRefRepo, 但 scheduler 路径的 DefaultSchedulerModeFactory 没注入。新实验类型 (MultiSetConfig) 首次调度触发 ExptSubmitExec.exptStartMultiSet 时, e.exptItemRefRepo 为 nil, 直接报错 "exptItemRefRepo is nil, cannot run multi-set ExptStart"。 BOE/PPE MQ consumer 已实锤复现, error stack 含 commit ff64c57 (PR2 之前 + 同事 commit, 两者都未含此修复)。 修复: - DefaultSchedulerModeFactory struct 加 exptItemRefRepo 字段; NewSchedulerModeFactory 加参数, factory NewSchedulerMode 内调用 NewExptSubmitMode / NewExptTrialRunMode 时传 f.exptItemRefRepo - NewExptTrialRunMode 加 variadic IExptItemRefRepo 透传给内部 NewExptSubmitMode (和后者 variadic 形式对齐) - wire 重新生成 wire_gen.go, NewSchedulerModeFactory 调用点 自动传入 iExptItemRefRepo - 测试 TestNewSchedulerModeFactory 补 MockIExptItemRefRepo 参数 FailRetry/Append/RetryAll/RetryItems struct 都没 exptItemRefRepo 字段, 不受影响, 本次只修 Submit + TrialRun 路径。 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… BAM parse domain/expt.thrift 同时 include data + observability 两个同名 filter.thrift, basename 相同导致 Thrift 别名冲突。thriftgo/kitex 能自动消歧(filter0),但 BAM 解析器把 filter.Filter 误绑到 observability namespace(无 Filter 结构)→ 'struct not defined' → 阻塞 BAM idl update / AGW 发布。 重命名 data 侧 filter.thrift → data_filter.thrift,引用方改用 data_filter.Filter, 消除同名歧义。namespace 不变 → Go 包路径不变 → 后端业务代码零改动。 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CallEvaluators 入口在 ItemConfig 非空时分流到 callEvaluatorsByItemConfig: - 主循环按 ItemConfig.EvaluatorConfs 遍历, 同 versionID 不同 alias 跑两次 - existResult 按 (versionID, alias) 双键复用 - filter 不命中 → 写 Skipped 占位 record (带 alias) - 命中 → 调 RunEvaluator/AsyncRunEvaluator, 把 alias + SourceType 透传 evaluator_impl.RunEvaluator/AsyncRunEvaluator 写入端把 request.Alias/SourceType 落到 EvaluatorRecord, 兜底 SourceType=Unknown -> Builtin. 清理 PR2 留下的 dead helper shouldRunEvaluatorByItemConfig + 单测, 该函数按 versionID 反查首个 conf, 在 alias 多实例下语义不正确; 现在 callEvaluatorsByItemConfig 直接按 conf 调 ShouldRunByFilter 不再需要它.
commit 2ced00e 把 submitReq := experiment.OpenAPITemplateToSubmitExperimentRequest(...) 误写为 expeinternal / cmd / evaluation / experiment.goriment.OpenAPITemplateToSubmitExperimentRequest(...), 导致 application 包整体不编译; 还原为原调用。
Get/MGet/List 实验响应里 eval_set_details[].item_count 此前一直是空。 本 PR 把读路径接通: - entity.Experiment 加 EvalSetDetails 字段 - entity 新增 ExptEvalSetDetail (与 IDL domain/expt.thrift ExptEvalSetDetail 同构) - ExptMangerImpl.NewExptManager 加 IExptItemRefRepo 参数 + struct 字段 - packExperimentResult 末尾调 fillEvalSetDetails (新实验类型才进, 老实验跳过) - fillEvalSetDetails: 按 EvalConf.EvalSetConfigs 拼骨架, IsPrimary 与 EvalSetID 一致, ItemCount 来源 IExptItemRefRepo.CountByEvalSetGrouped, 首跑前为 0 - DTO converter 把 entity.EvalSetDetails 转 thrift Experiment.EvalSetDetails - wire_gen 重新生成 5 个子测试 PASS: nil repo 跳过 / 仅 MultiSetConfig 触发 repo / IsPrimary 与 EvalSetID 一致 / ItemCount 缺失补 0 / repo 错误传播 / EvalConf 缺失跳过 / 无 MultiSetConfig 不调 repo。 全 evaluation 模块 go test ./... 0 回归。
新实验类型 MultiSetConfig 支持同评估器版本多别名(alias)，治理"按 evaluator_version_id 索引的 map 在 alias 多实例下撞 key"问题，覆盖聚合/ 加速数仓(CK)/导出三条旧消费链路。向前兼容：旧实验(alias 为空)编码退化为 裸 version_id，全链路 byte 级不变。 - 新增 entity.EncodeEvaluatorInstanceKey: alias="" 退化裸 verID，否则 verID:alias；与 ParseEvaluatorScoreFieldKey 互逆 - 聚合: 去掉 3 处 MultiSetConfig 跳过分支，恢复加权平均；computeEvaluator AggrGroup 按 (version,alias) 分桶；field_key 编码 verID:alias - 加权计算: CalculateWeightedScore 两 map key int64→string；新增类型感知 buildScoreWeights，MultiSetConfig 从 EvalSetConfigs[].EvaluatorConfs 带 alias 取权重 - CK 宽表: 新增 item_version_id 列(String DEFAULT '0'，含两处 init-sql)； key_mapping FromField/lookup 升级 verID:alias；CreateExpt + ManualUpsert 的 key_mapping 按 (version,alias) 实例展开(全局递增 ToKey + 去重) - 导出: ExportExptResult_ 入口按 EvalSetSourceType 拒绝 MultiSetConfig， 不动 DoExportCSV / 洞察分析 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s 筛选) 两处缺口: 1. GetExperimentsOApi 单实验 Get 走 OpenAPIExptDO2DTO, 此前未回填 item-centric 字段 (eval_set_source_type/eval_set_details/evaluators_concur_num/total_item_count); §5 的回填只加在 DomainExperimentDTO2OpenAPI (List/Create resp 路径)。本次在 OpenAPIExptDO2DTO 补回填, total_item_count 仅 MultiSetConfig 回显 (对齐 ToExptDTO)。 2. ListExperimentsOApi 的 eval_set_source_types 双层缺失: OpenAPI ExperimentFilterOption 缺字段 + OpenAPIExperimentFilterOptionDTO2Domain 未透传。本次 OpenAPI IDL 加 eval_set_source_types(field 2, 与 fuzzy_name 同级) + regen kitex_gen + 转换 string→int 枚举透传 + 修「只传 source_types 被判空返回 nil」。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
standard eval output 接口(MGetExperimentStandardEvalOutputs / ListExperimentStandardEvalOutputs)鉴权临时移除，改为直接放行， 用于 BOE 自测阶段；后续恢复 api_key 校验。同步更新单测断言。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
覆盖近 86 个 commit 中新增/大改的核心业务代码： - convertor: aggr_result / evaluation_set(_item) 转换器 - entity: target_record utility / evaluation_set_item ItemErrorType 补丁分支 - infra: mysql/convert/expt DO↔PO round-trip、agent_studio/sandbox_scheduler 占位实现 - service: ProvideNilItemCompletePublisher provider Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts: # backend/api/handler/coze/loop/apis/eval_open_apiservice.go # backend/api/handler/coze/loop/apis/experiment_service.go # backend/api/router/coze/loop/apis/coze.loop.apis.go # backend/api/router/coze/loop/apis/middleware.go # backend/modules/evaluation/application/convertor/experiment/aggr_result_test.go # backend/modules/evaluation/application/eval_openapi_app.go # backend/modules/evaluation/application/wire_gen.go # backend/modules/evaluation/domain/service/evaluator_impl_test.go # backend/modules/evaluation/domain/service/expt_run_item_impl.go # backend/modules/evaluation/domain/service/expt_run_item_turn_impl.go # backend/modules/evaluation/domain/service/expt_run_item_turn_impl_test.go
- convertor/evaluation_set: expected EvaluationSet 补齐 DatasetKey (do.DatasetKey 现已透传为 *string(""))
- convertor/experiment: ConvertCreateReq ItemConcurNum 断言改用 NormalizeSubmitItemConcurNum 归一化后比较
- domain/service:
- CompleteExpt terminated 分支补 MGetItemTurnRunLogs mock（沙箱销毁 best-effort 新增调用）
- BatchGetExptAggrResultByExperimentIDs mock experiment 补 SpaceID 以过通过越权过滤
- MGetExperimentResult 相关子测试补 BatchGetEvaluationSets / BatchGetEvaluationSetVersions mock（dataset_key 填充新增调用）
- asyncCallEvaluator: 失败时创建失败记录并存入 collector，与 sync 路径行为一致；相关测试改用 collector 断言
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SubmitExperiment/CreateExperiment 此前对调用方显式传入的 ExperimentGroupKey 不做任何校验(仅空值兜底用实验 ID),外部可传入 与已有实验重复的 group key。 改动: - domain/repo + infra/dao 新增 ExistGroupKey(groupKey): 全局(跨 space) 判断 group key 是否已被占用(COUNT LIMIT 1) - domain/service 新增 CheckGroupKey, 仿 CheckName - 创建主链路: 仅当调用方显式传入非空 group key 时查 DB 校验唯一, 撞车返回新错误码 ExperimentGroupKeyExistedCode(601204017); 未传时走"默认=实验ID"逻辑, 天然唯一, 不查 DB - 重新生成受影响的 repo/service/dao mocks 顺带修复 errno 源/产物漂移: pkg/errno/evaluation.go 是 script/errorx/evaluation.yaml 经 code_gen.py 生成的产物 (// Code generated by tool. DO NOT EDIT.), 但 3 个错误码 (AsyncEvaluatorZombieTimeout 601205069 / AsyncEvalTargetZombieTimeout 070 / AsyncEvalTargetTerminated 071)曾被手加进产物、未回写 YAML 源。 本次将其补回 evaluation.yaml, 使新错误码经正规 code_gen.py 流程生成, 源与产物重新一致。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
agent 块原先只拼 eval_target_id(=EvalTargetRecord.TargetID),下游拿不到 业务侧原始对象 ID。补 source_target_id:按 TargetID 反查 EvalTarget 得到 target 级恒定的 SourceTargetID(与 version 无关),MGet/List 两入口预解析 成 map 传入纯函数 builder。反查带 SpaceID 校验防跨空间越权,失败/跨空间/ 空串降级为不填,不阻断主链路。老字段 target_id/target_version_id 不动。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SandboxAgent 评测对象在 SubmitExperiment / RetryExperiment 时的沙箱 Init 由 fire-and-forget best-effort 改为同步阻塞：Init 失败直接向调用方返回错误， 避免后续 SandboxRun 依赖一个未成功初始化的沙箱任务。同步调整相关单测语义。 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
原 handler 只 bind 请求后返回空 response，从未调用 application 层， 导致该接口无论传入什么 group_key 都返回空 ExptIds，也不走鉴权与 DB 查询。 改为与同文件其它接口一致的 invokeAndRender，接通到 localExptSvc → application → domain service → repo 完整链路。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
单 item 评测完成事件 ItemCompleteEvent 原先只带 eval_target_id,下游拿不到 业务侧原始对象 ID。补 source_target_id:实验详情加载时 GetEvalTargetVersion 经 EvalTargetPO2DO 已回填 EvalTarget.SourceTargetID,组装处直接透传,零额外 IO。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- gofumpt/gofmt: 收敛格式化 (含 -extra 规则) - unconvert: 删除别名恒等类型转换 (data_tag/domain_filter/SandboxAgentType 等) - staticcheck QF1008: 简化 etec.ExptItemEvalCtx 嵌入字段选择 - unused: 删除未被引用的 runtimeParamFromTargetRecord / createAndStoreFailedEvaluatorRecord Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add filter(40, data_filter.Filter) + filter_mode(41, i32) to OpenAPIExptEvaluatorConf so OpenAPI SubmitExperiment can carry row-level (per-binding) evaluator filters, matching the internal ExptEvaluatorConf.filter/filter_mode semantics. Convertor passes them through to the internal domain; whitelist validation is still handled by the internal SubmitExperiment path. Regenerated kitex_gen and added TestOpenAPIEvalSetConfigsDTO2Domain_EvaluatorFilter. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
补齐 target/eval_target.go 中 SandboxAgent 相关分支的单测: - EvalTargetVersionDO2DTO 的 EvalTargetTypeSandboxAgent 分支 - SandboxAgentDTO2DO / SandboxAgentDO2DTO - SandboxEnvVarsDTO2DO / SandboxEnvVarsDO2DTO 覆盖 nil 输入、空切片、混杂 nil 元素、零值指针等边界。 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…filter Emit a compact per-conf decision log (run/skip + item.ItemID + owned_tags observed at runtime + filter fields/values) for every evaluator conf that carries a filter. Pinpoints whether a tag filter that culls all items is caused by item.Tags not being loaded (owned_tags empty) versus a values mismatch, without changing any culling semantics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…elds + ddl) Carry filter/filter_mode through the OpenAPI experiment submit path: IDL + generated kitex, app wiring, experiment model, and the experiment DDL/patch SQL for the new columns. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rmat buildExptTurnEvalCtx and pruneSuccessfulEvaluatorRecords only inspected the legacy EvalVerIDToResID map, but the persist path writes only the new Registered/Inline arrays. Async evaluator report re-triggers then saw empty EvaluatorResults and reported "evaluator result missing"; fail-retry likewise discarded already-successful records. Both now collect record IDs across Registered, Inline and the legacy map. Prune preserves the original shape so (VersionID, Alias) / InlineKey metadata is retained. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…val target 沙箱评测对象透传两个平台稳定标识到外部执行侧： - ExperimentGroupKey（实验级）经 ExecuteTargetCtx / ExecuteEvalTargetParam 透传 - DatasetKey（评测集级）填入 EvalSetItemMeta，来自 EvaluationSet.DatasetKey 新增字段均 optional，向前兼容。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- 新增 ref_group_experiment_id 字段(内部 Create/Submit + OpenAPI): 引用当前空间实验则复用其 group key 归入同组, 优先级高于 experiment_group_key - group key 唯一校验从全局改为跨空间隔离(space_id != 当前空间才算冲突), 同空间允许多实验共享同一 group - 新增错误码 RefGroupExperimentInvalid(601204019); ExperimentGroupKeyExisted 文案改为 already exists please use another one - 重新生成 kitex_gen / errno 产物 / repo·service·dao mock Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add Apache-2.0 header to standard_eval_output_test.go - Drop unnecessary string() conversion on QueryType (unconvert) - gofmt EvalSetItemMeta struct field alignment Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Check the PR title
(Optional) Translate the PR title into Chinese
(Optional) More detailed description for this PR(en: English/zh: Chinese)
en:
zh(optional):
(Optional) Which issue(s) this PR fixes