Skip to content

[Improvement-18593] [API&DAO] Optimize TaskInstanceMapper to exclude large text fields from list queries - #18595

Open
njnu-seafish wants to merge 2 commits into
apache:devfrom
njnu-seafish:Improvement-18593
Open

[Improvement-18593] [API&DAO] Optimize TaskInstanceMapper to exclude large text fields from list queries#18595
njnu-seafish wants to merge 2 commits into
apache:devfrom
njnu-seafish:Improvement-18593

Conversation

@njnu-seafish

Copy link
Copy Markdown
Contributor

Was this PR generated or assisted by AI?

YES. Leverage LLMs to verify that the removed large text fields are indeed unused.

Purpose of the pull request

close #18593

Brief change log

Optimize TaskInstanceMapper to exclude large text fields from list queries

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

@ruanwenjun ruanwenjun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please provide benchmark results to show exactly how much performance this type of refactoring can improve. Otherwise, we should return all fields in Listing interface, rather than add new VO/DTO.


private int pid;

private String appLink;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the text column at listing api.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the text column at listing api.

Sorry, I'm not quite sure I understand this review comment. I've already run the SQL benchmark with large fields excluded.

@njnu-seafish

Copy link
Copy Markdown
Contributor Author

Please provide benchmark results to show exactly how much performance this type of refactoring can improve. Otherwise, we should return all fields in Listing interface, rather than add new VO/DTO.

screenshot_1788242640938

Thanks for the feedback. We ran a benchmark on MySQL 8.0 (InnoDB) with 10,000 workflow instances and 50,000 task instances to quantify the improvement. The test simulates a typical list/paging query (LIMIT 20) and runs 10 iterations per variant.

Excluded columns:
screenshot_1788243108531

Benchmark results (10-run average, LIMIT 20 per page):
screenshot_1788243210881

Analysis:
SQL latency: t_ds_workflow_instance uses an index scan (start_time_index), so reducing the column set directly cuts the amount of row data read — yielding a 42.8% improvement. t_ds_task_instance does a full table scan + filesort (no covering index for project_code + submit_time), so the SQL execution time is dominated by sorting rather than column projection, and the latency gain is a more modest 9.3%.

Data transfer savings (not reflected in SQL latency): Each page of 20 rows avoids transferring ~64 KB and ~23 KB of large text respectively. In a real-world deployment, this overhead is amplified through the full stack: JDBC ResultSet materialization → MyBatis entity mapping → Jackson JSON serialization for the API response. For high-traffic list endpoints (e.g. task instance paging under a busy project), this reduction meaningfully lowers GC pressure and response payload size.

We believe the lightweight projection is worthwhile even for the t_ds_task_instance case: the 9.3% SQL latency improvement plus the ~23 KB/page transfer savings represent a real gain for a frequently accessed endpoint, with no functional impact (the omitted fields are only needed for task execution, detail views, or log retrieval, not for list/paging display).

@njnu-seafish njnu-seafish changed the title [Improvement] [API&DAO] Optimize TaskInstanceMapper to exclude large text fields from list queries [Improvement-18593] [API&DAO] Optimize TaskInstanceMapper to exclude large text fields from list queries Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] [API] Improve the task instance large-field optimization to align with the WorkflowInstance approach

2 participants