Lexical warnings - Perl 5 parity

dev/design/lexical-warnings.md

@@ -939,12 +939,196 @@ The following documents were superseded by this one and have been deleted:
 
 ## Progress Tracking
 
-### Status: Implementation Ready
+### Status: Phase 2 Complete (2026-03-29)
 
 ### Completed
 - [x] Design document created
 - [x] Superseded design documents deleted
+- [x] Phase 1: Infrastructure (2026-03-29)
+  - Created `WarningBitsRegistry.java` - HashMap registry for class name → warning bits
+  - Enhanced `WarningFlags.java`:
+    - Added `PERL5_OFFSETS` map with Perl 5 compatible category offsets
+    - Added `userCategoryOffsets` for `warnings::register` support
+    - Added `toWarningBitsString()` for caller()[9] bits format
+    - Added `isEnabledInBits()` and `isFatalInBits()` utility methods
+    - Added `registerUserCategoryOffset()` for dynamic category allocation
+  - Enhanced `ScopedSymbolTable.java`:
+    - Added `warningFatalStack` for FATAL warnings tracking
+    - Updated `enterScope()`/`exitScope()` to handle fatal stack
+    - Updated `snapShot()` and `copyFlagsFrom()` to copy fatal stack
+    - Added `enableFatalWarningCategory()`, `disableFatalWarningCategory()`, `isFatalWarningCategory()`
+    - Added `getWarningBitsString()` for caller()[9] support
+- [x] Phase 2: Two-variant operator methods (2026-03-29)
+  - Added `getNumberWarn(String operation)` to `RuntimeScalar.java`:
+    - Centralizes undef check and warning emission
+    - Correctly handles tied scalars (single FETCH)
+    - Returns scalarZero for UNDEF after emitting warning
+  - Added warn variants to `MathOperators.java`:
+    - `addWarn()` (both scalar,int and scalar,scalar)
+    - `subtractWarn()` (both variants)
+    - `multiplyWarn()`
+    - `divideWarn()`
+    - `modulusWarn()`
+    - `powWarn()`
+    - `unaryMinusWarn()`
+  - Refactored existing operators to remove inline warnings (fast path)
+  - Added warn operator entries to `OperatorHandler.java`:
+    - `+_warn`, `-_warn`, `*_warn`, `/_warn`, `%_warn`, `**_warn`, `unaryMinus_warn`
+  - Emitter already uses `OperatorHandler.getWarn()` based on `isWarningCategoryEnabled("uninitialized")`
+- [x] Phase 3: Per-closure warning bits storage for JVM backend (2026-03-29)
+  - Added `WarningBitsRegistry.java` in `org.perlonjava.runtime`:
+    - ConcurrentHashMap from class name to warning bits string
+    - `register()` method called from class static initializer
+    - `get()` method for caller() lookups
+    - `clear()` method for PerlLanguageProvider.resetAll()
+  - Updated `EmitterMethodCreator.java`:
+    - Added `WARNING_BITS` static final field to generated classes
+    - Added `<clinit>` static initializer to register bits with WarningBitsRegistry
+  - Updated `RuntimeCode.callerWithSub()`:
+    - Added `extractJavaClassNames()` helper to get Java class names from stack trace
+    - Element 9 now looks up warning bits from WarningBitsRegistry
+  - **Known Limitation**: Warning bits are per-class, not per-call-site
+    - Perl 5 tracks warning bits at statement granularity
+    - PerlOnJava tracks at class (closure) granularity
+    - All calls from the same class share the same warning bits
+    - Different closures DO get their own warning bits (correctly)
+- [x] Phase 4: Per-closure warning bits storage for interpreter (2026-03-29)
+  - Added `warningBitsString` field to `InterpretedCode.java`:
+    - Stores Perl 5 compatible warning bits string
+    - Passed from BytecodeCompiler using symbolTable.getWarningBitsString()
+  - Updated constructors in `InterpretedCode.java`:
+    - Main constructor accepts warningBitsString parameter
+    - Registers with WarningBitsRegistry using "interpreter:" + identityHashCode key
+    - withCapturedVars() copies warningBitsString to new instance
+  - Updated `BytecodeCompiler.buildInterpretedCode()`:
+    - Extracts warningBitsString from emitterContext.symbolTable
+    - Passes to InterpretedCode constructor
+  - `extractJavaClassNames()` in RuntimeCode already handles interpreter frames
+    - Uses "interpreter:" + System.identityHashCode(frame.code()) as registry key
+- [x] Phase 6: warnings:: functions using caller()[9] (2026-03-29)
+  - Updated `Warnings.java`:
+    - Added `getWarningBitsAtLevel()` helper to get warning bits from caller()
+    - `enabled()` now uses caller()[9] with `WarningFlags.isEnabledInBits()`
+    - `warnif()` now checks caller()[9] and handles FATAL warnings
+    - Added `fatal_enabled()` using `WarningFlags.isFatalInBits()`
+    - Added `enabled_at_level()` for checking at specific stack levels
+    - Added `fatal_enabled_at_level()` for FATAL check at specific levels
+    - Added `warnif_at_level()` for warning at specific stack levels
+  - Registered new methods in initialize():
+    - `warnings::enabled_at_level`, `warnings::fatal_enabled`
+    - `warnings::fatal_enabled_at_level`, `warnings::warnif_at_level`
 
 ### Next Steps
-1. Implement Phase 1: Infrastructure
-2. Continue with remaining phases
+
+#### Phase 9: Per-Call-Site Warning Bits (Future)
+
+**Goal:** Enable block-scoped `use warnings` / `no warnings` to work correctly.
+
+**Current Limitation:**
+Warning bits are captured per-class at compile time. This means:
+```perl
+sub foo {
+    my $x;
+    print $x . "a";  # Uses class-level warning bits
+    {
+        no warnings 'uninitialized';
+        print $x . "b";  # Still uses class-level bits - warns incorrectly!
+    }
+}
+```
+
+**Proposed Solution:**
+Store warning bits per-statement (call-site) rather than per-class.
+
+**Implementation Approach:**
+
+1. **Compile-time: Emit warning bits with each statement**
+   - Each statement that can warn stores its warning bits as a parameter
+   - Example: `concatWarn(a, b, warningBits)` instead of `concatWarn(a, b)`
+   - The `warningBits` is a compile-time constant string
+
+2. **Runtime: Check bits at call site**
+   - Warning operators receive the bits as a parameter
+   - `warnWithCategory()` uses the passed bits instead of looking up caller()
+   - No ThreadLocal or caller() lookup needed for most cases
+
+3. **Alternative: Scope ID approach**
+   - Each scope gets a unique ID at compile time
+   - Store `scopeId → warningBits` mapping in registry
+   - Emit `local ${^WARNING_SCOPE} = scopeId` at scope entry
+   - Runtime looks up bits by current scope ID
+
+**Trade-offs:**
+
+| Approach | Pros | Cons |
+|----------|------|------|
+| Per-statement bits | Fast, no lookup | Increases bytecode size |
+| Scope ID registry | Smaller bytecode | Runtime lookup overhead |
+
+**Files to Modify:**
+- `EmitOperator.java` - Pass warning bits to warn variants
+- `StringOperators.java` (and others) - Accept bits parameter
+- `WarnDie.java` - Use passed bits instead of caller() lookup
+- `ScopedSymbolTable.java` - Track scope-level warning changes
+
+**Estimated Complexity:** Medium-High
+- Requires changes to operator signatures
+- Need to update all warn-variant operators
+- Must maintain backward compatibility
+
+**Priority:** Low (current implementation handles most use cases)
+
+### Phase 7-8 Progress (2026-03-29)
+- [x] Added `warnWithCategory()` to WarnDie.java:
+  - Checks if warning category is FATAL in caller's scope
+  - Uses caller()[9] for subroutine frames
+  - Falls back to ThreadLocal context stack for top-level code
+  - Converts warning to die() when FATAL bit is set
+- [x] Added ThreadLocal context stack to WarningBitsRegistry:
+  - `pushCurrent()` / `popCurrent()` track current warning bits during execution
+  - `getCurrent()` retrieves bits for FATAL checks
+- [x] Updated RuntimeCode.apply() to push/pop warning bits
+- [x] Updated InterpretedCode.apply() to push/pop warning bits
+- [x] Updated StringOperators.stringConcatWarnUninitialized() to use warnWithCategory()
+
+**FATAL warnings work for:**
+- File-scope `use warnings FATAL => 'all'`
+- Named subroutines inheriting FATAL from enclosing scope
+- Top-level code execution
+
+**Known limitation:**
+Block-scoped `use warnings FATAL` inside a subroutine/program doesn't work because
+warning bits are captured per-class at compile time, not per-scope. This would require
+per-call-site warning bits for full parity.
+
+### Phase 8: $^W Interaction (2026-03-29)
+- [x] Added `isWarnFlagSet()` helper in Warnings.java:
+  - Checks if `$^W` global variable is set to a true value
+  - `$^W` is stored as `main::` + char(23) using Perl's special variable encoding
+- [x] Updated `warnif()` to fall back to `$^W`:
+  - If category is NOT enabled in lexical warnings, check `$^W`
+  - If `$^W` is true, issue warning
+  - This allows `$^W` to work with modules using `warnings::warnif()`
+- [x] Updated `warnIfAtLevel()` with same `$^W` fallback logic
+
+**$^W interaction works for:**
+- File-scope code without `use warnings` or `no warnings`
+- Module code calling `warnings::warnif()` when caller has `$^W = 1`
+
+**Known limitation:**
+Block-scoped `no warnings` doesn't override `$^W` for `warnif()` calls because
+our warning bits are per-class, not per-scope. This differs from Perl 5 where
+`no warnings` takes precedence over `$^W`. However, file-scope `no warnings`
+at the class level does correctly suppress warnings.
+
+**Test results:**
+```perl
+# Works correctly:
+$^W = 0; warnings::warnif("cat", "msg");  # No warning
+$^W = 1; warnings::warnif("cat", "msg");  # Warning issued
+use warnings; warnings::warnif("cat", "msg");  # Warning issued (file-scope)
+
+# Known limitation:
+$^W = 1;
+{ no warnings; warnings::warnif("cat", "msg"); }  # Warning issued (differs from Perl 5)
+```

docs/about/changelog.md

@@ -9,6 +9,7 @@ Release history of PerlOnJava. See [Roadmap](roadmap.md) for future plans.
 - Tools: added `jcpan`, `jperldoc`, and `jprove`
 - Perl debugger with `-d` command line option
 - Add `defer` feature
+- Lexical warnings with `use warnings` and FATAL support
 - Non-local control flow: `last`/`next`/`redo`/`goto LABEL`
 - Tail call with trampoline for `goto &NAME` and `goto __SUB__`
 - Add modules: `CPAN`, `Time::Piece`, `TOML`, `DirHandle`, `Dumpvalue`, `Sys::Hostname`, `IO::Socket`, `IO::Socket::INET`, `IO::Socket::UNIX`, `IO::Zlib`, `Archive::Tar`, `Archive::Zip`, `Net::FTP`, `Net::Cmd`, `IPC::Open2`, `IPC::Open3`, `ExtUtils::MakeMaker`.

docs/reference/feature-matrix.md

@@ -68,7 +68,7 @@ PerlOnJava implements most core Perl features with some key differences:
 - ✅  **Perl-like runtime error messages**: Runtime errors are formatted similarly to Perl's.
 - ✅  **Comments**: Support for comments and POD (documentation) in code is implemented.
 - ✅  **Environment**: Support for `PERL5LIB`, `PERL5OPT` environment variables.
-- 🚧  **Perl-like warnings**: Warnings is work in progress. Some warnings need to be formatted to resemble Perl's output.
+- 🚧  **Perl-like warnings**: Lexical warnings with FATAL support. Block-scoped warnings pending.
 
 ---
 

src/main/java/org/perlonjava/app/cli/ArgumentParser.java

@@ -324,8 +324,8 @@ private static int processClusteredSwitches(String[] args, CompilerOptions parse
                     return index;
 
                 case 'w':
-                    // enable many useful warnings
-                    parsedArgs.moduleUseStatements.add(new ModuleUseStatement(switchChar, "warnings", null, false));
+                    // enable many useful warnings by setting $^W = 1
+                    parsedArgs.warnFlag = true;
                     break;
                 case 'W':
                     // enable all warnings

src/main/java/org/perlonjava/app/cli/CompilerOptions.java

@@ -82,6 +82,7 @@ public class CompilerOptions implements Cloneable {
     public boolean unicodeOutput = false; // -CO (same as stdout)
     public boolean unicodeArgs = false; // -CA
     public boolean unicodeLocale = false; // -CL
+    public boolean warnFlag = false; // For -w (sets $^W = 1)
     public RuntimeScalar incHook = null; // For storing @INC hook reference
     List<ArgumentParser.ModuleUseStatement> moduleUseStatements = new ArrayList<>(); // For -m -M
 

src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java

@@ -245,6 +245,19 @@ public static RuntimeList executePerlAST(Node ast,
         // can see and modify the enclosing scope's compile-time hints
         if (savedCurrentScope != null) {
             globalSymbolTable.setStrictOptions(savedCurrentScope.getStrictOptions());
+            // Inherit warning flags so ${^WARNING_BITS} returns correct values in BEGIN blocks
+            if (!savedCurrentScope.warningFlagsStack.isEmpty()) {
+                globalSymbolTable.warningFlagsStack.pop();
+                globalSymbolTable.warningFlagsStack.push((java.util.BitSet) savedCurrentScope.warningFlagsStack.peek().clone());
+            }
+            if (!savedCurrentScope.warningDisabledStack.isEmpty()) {
+                globalSymbolTable.warningDisabledStack.pop();
+                globalSymbolTable.warningDisabledStack.push((java.util.BitSet) savedCurrentScope.warningDisabledStack.peek().clone());
+            }
+            if (!savedCurrentScope.warningFatalStack.isEmpty()) {
+                globalSymbolTable.warningFatalStack.pop();
+                globalSymbolTable.warningFatalStack.push((java.util.BitSet) savedCurrentScope.warningFatalStack.peek().clone());
+            }
         }
 
         EmitterContext ctx = new EmitterContext(

src/main/java/org/perlonjava/backend/bytecode/BytecodeCompiler.java

@@ -280,6 +280,8 @@ private void enterScope() {
             st.strictOptionsStack.push(st.strictOptionsStack.peek());
             st.featureFlagsStack.push(st.featureFlagsStack.peek());
             st.warningFlagsStack.push((java.util.BitSet) st.warningFlagsStack.peek().clone());
+            st.warningFatalStack.push((java.util.BitSet) st.warningFatalStack.peek().clone());
+            st.warningDisabledStack.push((java.util.BitSet) st.warningDisabledStack.peek().clone());
         }
     }
 
@@ -297,6 +299,8 @@ private void exitScope() {
                 st.strictOptionsStack.pop();
                 st.featureFlagsStack.pop();
                 st.warningFlagsStack.pop();
+                st.warningFatalStack.pop();
+                st.warningDisabledStack.pop();
             }
         }
     }
@@ -528,6 +532,10 @@ public InterpretedCode compile(Node node, EmitterContext ctx) {
                 symbolTable.featureFlagsStack.push(ctx.symbolTable.featureFlagsStack.peek());
                 symbolTable.warningFlagsStack.pop();
                 symbolTable.warningFlagsStack.push((java.util.BitSet) ctx.symbolTable.warningFlagsStack.peek().clone());
+                symbolTable.warningFatalStack.pop();
+                symbolTable.warningFatalStack.push((java.util.BitSet) ctx.symbolTable.warningFatalStack.peek().clone());
+                symbolTable.warningDisabledStack.pop();
+                symbolTable.warningDisabledStack.push((java.util.BitSet) ctx.symbolTable.warningDisabledStack.peek().clone());
             }
         }
 
@@ -577,10 +585,12 @@ public InterpretedCode compile(Node node, EmitterContext ctx) {
         int strictOptions = 0;
         int featureFlags = 0;
         BitSet warningFlags = new BitSet();
+        String warningBitsString = null;
         if (emitterContext != null && emitterContext.symbolTable != null) {
             strictOptions = emitterContext.symbolTable.strictOptionsStack.peek();
             featureFlags = emitterContext.symbolTable.featureFlagsStack.peek();
             warningFlags = (BitSet) emitterContext.symbolTable.warningFlagsStack.peek().clone();
+            warningBitsString = emitterContext.symbolTable.getWarningBitsString();
         }
 
         // Populate debug source lines if in debug mode
@@ -608,7 +618,8 @@ public InterpretedCode compile(Node node, EmitterContext ctx) {
                 warningFlags,
                 symbolTable.getCurrentPackage(),
                 evalSiteRegistries.isEmpty() ? null : evalSiteRegistries,
-                evalSitePragmaFlags.isEmpty() ? null : evalSitePragmaFlags
+                evalSitePragmaFlags.isEmpty() ? null : evalSitePragmaFlags,
+                warningBitsString
         );
         // Set optimization flag - if no LOCAL_* or PUSH_LOCAL_VARIABLE opcodes were emitted,
         // the interpreter can skip DynamicVariableManager.getLocalLevel/popToLocalLevel
@@ -5286,6 +5297,10 @@ public void visit(CompilerFlagNode node) {
             ScopedSymbolTable st = emitterContext.symbolTable;
             st.warningFlagsStack.pop();
             st.warningFlagsStack.push((java.util.BitSet) node.getWarningFlags().clone());
+            st.warningFatalStack.pop();
+            st.warningFatalStack.push((java.util.BitSet) node.getWarningFatalFlags().clone());
+            st.warningDisabledStack.pop();
+            st.warningDisabledStack.push((java.util.BitSet) node.getWarningDisabledFlags().clone());
             st.featureFlagsStack.pop();
             st.featureFlagsStack.push(node.getFeatureFlags());
             st.strictOptionsStack.pop();
@@ -5297,6 +5312,10 @@ public void visit(CompilerFlagNode node) {
         symbolTable.strictOptionsStack.push(node.getStrictOptions());
         symbolTable.warningFlagsStack.pop();
         symbolTable.warningFlagsStack.push((java.util.BitSet) node.getWarningFlags().clone());
+        symbolTable.warningFatalStack.pop();
+        symbolTable.warningFatalStack.push((java.util.BitSet) node.getWarningFatalFlags().clone());
+        symbolTable.warningDisabledStack.pop();
+        symbolTable.warningDisabledStack.push((java.util.BitSet) node.getWarningDisabledFlags().clone());
 
         lastResultReg = -1;
     }

src/main/java/org/perlonjava/backend/bytecode/CompileAssignment.java

@@ -1,5 +1,6 @@
 package org.perlonjava.backend.bytecode;
 
+import org.perlonjava.frontend.analysis.ConstantFoldingVisitor;
 import org.perlonjava.frontend.analysis.LValueVisitor;
 import org.perlonjava.frontend.astnode.*;
 import org.perlonjava.runtime.runtimetypes.NameNormalizer;
@@ -1549,6 +1550,20 @@ public static void compileAssignmentOperator(BytecodeCompiler bytecodeCompiler,
                     return;
                 }
 
+                // Handle constant-folded logical operators: e.g. `1 && my $x = val` → `my $x = val`
+                // Perl constant-folds logical ops with constant LHS at compile time.
+                if (leftBin.operator.equals("&&") || leftBin.operator.equals("and") ||
+                        leftBin.operator.equals("||") || leftBin.operator.equals("or") ||
+                        leftBin.operator.equals("//")) {
+                    Node foldedLeft = ConstantFoldingVisitor.foldConstants(node.left);
+                    if (foldedLeft != node.left) {
+                        // Operator was folded - recursively handle assignment with folded LHS
+                        BinaryOperatorNode newNode = new BinaryOperatorNode("=", foldedLeft, node.right, node.tokenIndex);
+                        compileAssignmentOperator(bytecodeCompiler, newNode);
+                        return;
+                    }
+                }
+
                 bytecodeCompiler.throwCompilerException("Assignment to non-identifier not yet supported: " + node.left.getClass().getSimpleName());
             } else if (node.left instanceof TernaryOperatorNode) {
                 LValueVisitor.getContext(node.left);