Skip to content

TestRail Deduplication: matrix strategy, BQ tracking, and Slack notifications#323

Open
mdotb-moz wants to merge 19 commits intomainfrom
mb/MTE-4838-deduplication-ios
Open

TestRail Deduplication: matrix strategy, BQ tracking, and Slack notifications#323
mdotb-moz wants to merge 19 commits intomainfrom
mb/MTE-4838-deduplication-ios

Conversation

@mdotb-moz
Copy link
Copy Markdown
Collaborator

@mdotb-moz mdotb-moz commented Apr 23, 2026

Summary

This PR implements the github actions to run the a local model to find test duplications from a test rail export.

  • Matrix strategy: runs deduplication in parallel for all 4 projects (firefox-ios, fenix, focus-ios, focus-android) using a single workflow definition
  • BigQuery tracking: new testrail_deduplication_runs table records weekly stats per project; workflow queries previous week's data to compute deltas
  • Slack notifications: weekly digest with delta vs last week, spike alert when exact duplicates jump by >10; failure alert on error
  • New script insert_bq_stats.py: reads output CSVs safely, writes stats to GITHUB_ENV, inserts into BigQuery — replaces brittle inline Python/shell in YAML

mdotb-moz and others added 13 commits April 21, 2026 10:44
…to Python script

- Move TESTRAIL_HOST/USERNAME/PASSWORD env vars from job level to the fetch
  step only, limiting credential exposure to the single step that needs them
- Replace inline shell/Python BigQuery insertion with insert_bq_stats.py,
  eliminating JSON injection risk from shell string interpolation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mdotb-moz mdotb-moz changed the title TestRail Deduplication: matrix strategy, BQ tracking, and Slack notifications WIP: TestRail Deduplication: matrix strategy, BQ tracking, and Slack notifications Apr 23, 2026
@mdotb-moz mdotb-moz requested review from clarmso and isabelrios April 23, 2026 09:33

- name: Build Slack payloads
run: |
python3 << 'PYEOF'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we could move things like this to separate files to not have this long github action and specific scripts are isolated rather than in the action themselves

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I moved this into a separate python file

print("="*60)

if __name__ == "__main__":
import argparse
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not this live above with the rest of imports?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right

import re
import unicodedata
from difflib import SequenceMatcher

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: space either above or remove

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the space

## FAQs

### Q: Which test should I keep if they're all identical?
**A:** Keep the one with the **lowest Case ID** (e.g., C2575167 instead of C3193560). The lowest ID is usually the original test, and higher IDs are copies.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about that.. I will keep the one that is linked in githug to avoid creating a PR just to modify the link

mdotb-moz and others added 4 commits April 24, 2026 10:54
Replaces the PYEOF heredoc in the workflow with a proper Python script,
keeping the YAML clean and making the logic testable independently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t-priority-list.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mdotb-moz mdotb-moz changed the title WIP: TestRail Deduplication: matrix strategy, BQ tracking, and Slack notifications TestRail Deduplication: matrix strategy, BQ tracking, and Slack notifications Apr 24, 2026
@mdotb-moz mdotb-moz requested a review from isabelrios April 24, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants