Skip to content

Prevent variable nullification on later imports - #124

Merged
andrus merged 1 commit into
dflib:mainfrom
m-dzianishchyts:119-fix-variable-not-updating-on-redeclaration
Aug 17, 2026
Merged

Prevent variable nullification on later imports#124
andrus merged 1 commit into
dflib:mainfrom
m-dzianishchyts:119-fix-variable-not-updating-on-redeclaration

Conversation

@m-dzianishchyts

@m-dzianishchyts m-dzianishchyts commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Prevents newlines from causing unnecessary JShell wrapper redefinitions and losing previously initialized variables.

The normalization is applied immediately before each snippet is evaluated by JShell, while the original source remains available for completion analysis and snippet processing.

Details

JShell preserves the newline separating snippets when returning CompletionInfo.source(). Passing that leading newline directly to JShell changes the generated wrapper source layout and its class file line-number metadata.

For imports that do not alter executable code, this creates a bytecode difference that incorrectly triggers ExecutionControl.redefine(). Since JJavaExecutionControl does not support in-place redefinition, JShell creates a replacement wrapper with default-initialized fields, causing previously assigned variables to become null.

Stripping surrounding whitespace characters before evaluation removes the accidental source-layout difference while preserving the actual import and code semantics. Legitimate changes to executable bytecode remain unaffected.

The regression test for later imports is enabled, and coverage for variable redeclaration remains intact.

Testing

Verified with existing integration tests and manual testing in Jupyter Notebook.

@m-dzianishchyts
m-dzianishchyts marked this pull request as ready for review June 24, 2026 16:00
@m-dzianishchyts
m-dzianishchyts marked this pull request as draft June 26, 2026 12:36
@m-dzianishchyts m-dzianishchyts changed the title Implement JVM HotSwap via Java Agent Revert #120 "Prevent variable nullification on later imports" Jul 6, 2026
@m-dzianishchyts
m-dzianishchyts marked this pull request as ready for review July 17, 2026 12:47
@andrus

andrus commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

@m-dzianishchyts , @stariy95 : guys, I tried to understand how this PR is different from the original #120. So I manually reverted all #120 commits (already pushed to "main") and then did a merge --squash of this one on top. The only change that I see is introduction of KernelExecutionIT.java test. But the original problem is still there when I run it manually. It was not fixed.

@andrus

andrus commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Ok, I was able to rewrite KernelExecutionIT in a way that reproduces the issue. It is committed to main (see a @Disabled test). The reason the original test didn't fail was that each line was executed in a separate cell. Somehow combining things in cells in a certain way causes this problem. Here are Claude findings on which combos work and which don't:

  ┌──────────────────────────────────────────────────────────┬────────┐
  │                          cells                           │ result │
  ├──────────────────────────────────────────────────────────┼────────┤
  │ 5 separate cells (what the IT does)                      │ ✅ ok  │
  ├──────────────────────────────────────────────────────────┼────────┤
  │ all 5 in one cell                                        │ ❌ NPE │
  ├──────────────────────────────────────────────────────────┼────────┤
  │ %maven alone, then remaining 4 lines in one cell         │ ❌ NPE │
  ├──────────────────────────────────────────────────────────┼────────┤
  │ %maven / import+var om+import / om.getClass()            │ ❌ NPE │
  ├──────────────────────────────────────────────────────────┼────────┤
  │ %maven / import+var om / import+om.getClass()            │ ✅ ok  │
  └──────────────────────────────────────────────────────────┴────────┘

One little thing left - figure out how to fix it 🙂

@m-dzianishchyts
m-dzianishchyts force-pushed the 119-fix-variable-not-updating-on-redeclaration branch from 88e3b90 to 3e1ca69 Compare August 17, 2026 08:54
@m-dzianishchyts m-dzianishchyts changed the title Revert #120 "Prevent variable nullification on later imports" Prevent variable nullification on later imports Aug 17, 2026
@m-dzianishchyts

Copy link
Copy Markdown
Contributor Author

Hi, @andrus! There is one more clue, requesting review 👋

@andrus
andrus merged commit 3e1ca69 into dflib:main Aug 17, 2026
5 checks passed
@andrus

andrus commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

It works for me now. Amazing how trimming the code would make a difference. Thanks for digging it up! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants