Out memory when attempting to open VSCode

[kchanqvq]

I installed latest VS Code (from official .deb) on Ubuntu, with latest EXWM (3b861e2). EXWM freezes then Emacs get killed as soon as I $ code.

$ uname -a
Linux ... #19~24.04.2-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar  6 23:08:46 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
$ code --version
1.113.0
cfbea10c5ffb233ea9177d34726e6056e89913dc
x64

Mar 26 23:29:09 ... kernel: Out of memory: Killed process 6792 (emacs) total-vm:19079752kB, anon-rss:14908548kB, file-rss:448kB, shmem-rss:0kB, UID:1000 pgtables:36312kB oom_score_adj:0

I'm quite clueless, happy to provide other diagnostics if I'm told what to look for. Does anyone else experience the same problem?

[Stebalien]

Are you using XIM?

[kchanqvq]

Are you using XIM?

No, I don't use an input method, and GTK_IM_MODULE=ibus.

[kchanqvq]

I tried to avoid managing it via

(setq-default exwm-manage-configurations
                '(((string= "code" exwm-instance-name) managed nil)))

but this seems to have no effect. Either I'm doing it wrong or the problem occur before the above variable take effect?

[kchanqvq]

Setting emacs.useXIM: false does not help either.

(x-get-resource "useXIM" "UseXIM")
"false"

[Stebalien]

Just to triple check, exwm-xim-mode is false?

In that case, at least we've eliminated some of the hardest to debug code...

Can you:

1. Enable exwm-debug-mode.
2. Open a shell in one window and the *XELB-DEBUG* buffer in another window.
3. Launch vscode from the shell.
4. Somehow try to capture what you see in *XELB-DEBUG* before everything crashes (maybe a video on a phone?).

It's either:

1. Some kind of infinite loop between vscode and EXWM that blows up memory.
2. A single massive allocation, likely because vscode is returning an invalid length for something (e.g., -1).

My guess is that the issue is (2), which may make this hard to debug without instrumenting XELB.

[kchanqvq]

Just to triple check, exwm-xim-mode is false?

Right, it is nil.

I'm able to break out of the OOM freeze by holding quit, although after that it seems to leave EXWM in some inconsistent state (glitchy window, input sometimes don't work...) but I do get the *XELB-DEBUG* output:

[00:00:29]exwm-input--update-focus:	focus-window=#<window 3 on *vterminal<1>*> focus-buffer=*vterminal<1>*
[00:00:29]exwm-input--update-focus:	Focus on #<window 3 on *vterminal<1>*>
[00:00:29]exwm-input--set-active-window:	
[00:00:31]exwm-manage--on-ConfigureRequest:	
[00:00:31]exwm-manage--on-ConfigureRequest:	#x1000002 (#xf) @20x20-200-200; border-width: 0; sibling: #x0; stack-mode: 0
[00:00:31]exwm-manage--on-ConfigureRequest:	ConfigureWindow (preserve geometry)
[00:00:31]exwm-manage--on-ConfigureRequest:	
[00:00:31]exwm-manage--on-ConfigureRequest:	#x1400000 (#xf) @20x20-200-200; border-width: 0; sibling: #x0; stack-mode: 0
[00:00:31]exwm-manage--on-ConfigureRequest:	ConfigureWindow (preserve geometry)
[00:00:32]exwm-manage--on-ConfigureRequest:	
[00:00:32]exwm-manage--on-ConfigureRequest:	#x1400000 (#xf) @20x20-200-200; border-width: 0; sibling: #x0; stack-mode: 0
[00:00:32]exwm-manage--on-ConfigureRequest:	ConfigureWindow (preserve geometry)

== I think it freezes here, and I start holding C-g. More outputs show up once it unfreezes ==

[00:00:32]exwm-manage--on-MapRequest:	id=#x1000003 parent=#x4b6
[00:00:32]exwm-manage--on-MapRequest:	#x1000003
[00:00:32]exwm-manage--manage-window:	Try to manage #x1000003
[00:00:32]exwm--update-window-type:	#x1000003
[00:00:32]exwm--update-class:	#x1000003
[00:00:32]exwm--update-transient-for:	#x1000003
[00:00:32]exwm--update-normal-hints:	#x1000003
[00:00:32]exwm--update-hints:	#x1000003
[00:00:32]exwm-manage--update-geometry:	id=#x1000003
[00:00:32]exwm-manage--update-mwm-hints:	id=#x1000003
[00:00:32]exwm--update-title:	#x1000003
[00:00:32]exwm--update-utf8-title:	#x1000003
[00:00:32]exwm--update-ctext-title:	#x1000003
[00:00:32]exwm--update-icon:	#x1000003
[00:00:35]exwm-input--on-buffer-list-update:	current-buffer=#<buffer *vterminal<1>*> selected-window=#<window 3 on *vterminal<1>*>
[00:00:35]exwm--on-ClientMessage:	atom=_NET_ACTIVE_WINDOW(367) id=#x1000003 data=(1 10461890 0 0 0)
[00:00:35]exwm-input--on-buffer-list-update:	current-buffer=#<buffer Code: Visual Studio Code> selected-window=#<window 3 on *vterminal<1>*>
[00:00:35]exwm-input--on-buffer-list-update:	current-buffer=#<buffer Code: Visual Studio Code> selected-window=#<window 3 on Code: Visual Studio Code>
[00:00:36]exwm-input--update-focus:	focus-window=#<window 3 on Code: Visual Studio Code> focus-buffer=Code: Visual Studio Code
[00:00:36]exwm-input--update-focus:	Set focus on #x1000003
[00:01:29]exwm--on-ClientMessage:	atom=_NET_ACTIVE_WINDOW(367) id=#x1000003 data=(1 10461891 0 0 0)
[00:01:29]exwm-input--on-buffer-list-update:	current-buffer=#<buffer Code: Visual Studio Code> selected-window=#<window 3 on Code: Visual Studio Code>
[00:01:29]exwm--on-ClientMessage:	atom=_NET_WM_STATE(378) id=#x1000003 data=(0 379 0 1 0)
[00:01:29]exwm-manage--on-ConfigureRequest:	
[00:01:29]exwm-manage--on-ConfigureRequest:	#x1600000 (#xf) @20x20-200-200; border-width: 0; sibling: #x0; stack-mode: 0
[00:01:29]exwm-manage--on-ConfigureRequest:	ConfigureWindow (preserve geometry)
[00:01:29]exwm-manage--on-ConfigureRequest:	
[00:01:29]exwm-manage--on-ConfigureRequest:	#x1600000 (#xf) @20x20-200-200; border-width: 0; sibling: #x0; stack-mode: 0
[00:01:29]exwm-manage--on-ConfigureRequest:	ConfigureWindow (preserve geometry)
[00:01:29]exwm-manage--on-ConfigureRequest:	
[00:01:29]exwm-manage--on-ConfigureRequest:	#x1800001 (#xf) @20x20-200-200; border-width: 0; sibling: #x0; stack-mode: 0
[00:01:29]exwm-manage--on-ConfigureRequest:	ConfigureWindow (preserve geometry)
[00:01:29]exwm--on-PropertyNotify:	atom=_NET_WM_NAME(375)
[00:01:29]exwm--update-utf8-title:	#x1000003
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm--on-PropertyNotify:	atom=WM_NAME(39)
[00:01:29]exwm--update-ctext-title:	#x1000003
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm--on-PropertyNotify:	atom=_NET_WM_STATE(378)
[00:01:29]exwm--on-PropertyNotify:	Unhandled: _NET_WM_STATE(378)
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm-input--set-focus:	#xc0002b (corrected) => #x1000003
[00:01:29]exwm-input--update-timestamp:	
[00:01:29]exwm--on-PropertyNotify:	atom=_TIME(576)
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm-input--set-active-window:	
[00:01:29]exwm-input--update-focus:	focus-window=#<window 3 on Code: Welcome - Visual Studio Code> focus-buffer=Code: Welcome - Visual Studio Code
[00:01:29]exwm-input--update-focus:	Set focus on #x1000003
[00:01:29]exwm-input--set-focus:	#xc0002b (corrected) => #x1000003
[00:01:29]exwm-input--update-timestamp:	
[00:01:29]exwm-layout--refresh:	frame=#<frame Code: Welcome - Visual Studio Code - GNU Emacs at kchan-ThinkPad-X1-Carbon-6th 0x5812cbd474c0>
[00:01:29]exwm-layout--refresh-workspace:	Refresh workspace #<frame Code: Welcome - Visual Studio Code - GNU Emacs at kchan-ThinkPad-X1-Carbon-6th 0x5812cbd474c0>
[00:01:29]exwm-layout--show:	Show #x1000003 in #<window 3 on Code: Welcome - Visual Studio Code>
[00:01:29]exwm--set-geometry:	Setting #x1000003 to 1220x1330+20+20
[00:01:29]exwm-layout--set-state:	id=#x1000003
[00:01:29]exwm-layout--set-client-list-stacking:	
[00:01:29]exwm-layout--refresh:	frame=#<frame Code: Welcome - Visual Studio Code - GNU Emacs at kchan-ThinkPad-X1-Carbon-6th 0x5812cbd474c0>
[00:01:29]exwm-layout--refresh-workspace:	Refresh workspace #<frame Code: Welcome - Visual Studio Code - GNU Emacs at kchan-ThinkPad-X1-Carbon-6th 0x5812cbd474c0>
[00:01:29]exwm-layout--show:	Show #x1000003 in #<window 3 on Code: Welcome - Visual Studio Code>
[00:01:29]exwm--set-geometry:	Setting #x1000003 to 1220x1330+20+20
[00:01:29]exwm-layout--set-state:	id=#x1000003
[00:01:29]exwm-layout--set-client-list-stacking:	
[00:01:29]exwm--on-PropertyNotify:	atom=_TIME(576)
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm-input--set-active-window:	
[00:01:29]exwm-manage--on-MapNotify:	id=#x1000003
[00:01:29]exwm--on-PropertyNotify:	atom=WM_STATE(366)
[00:01:29]exwm--on-PropertyNotify:	Unhandled: WM_STATE(366)
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm--on-PropertyNotify:	atom=_NET_WM_STATE(378)
[00:01:29]exwm--on-PropertyNotify:	Unhandled: _NET_WM_STATE(378)
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm--on-PropertyNotify:	atom=WM_STATE(366)
[00:01:29]exwm--on-PropertyNotify:	Unhandled: WM_STATE(366)
[00:01:29]exwm-input--on-PropertyNotify:	
[00:01:29]exwm--on-PropertyNotify:	atom=_NET_WM_STATE(378)
[00:01:29]exwm--on-PropertyNotify:	Unhandled: _NET_WM_STATE(378)
[00:01:29]exwm-input--on-PropertyNotify:	

[Stebalien]

Nothing in there looks unusual. Can you try enabling debug-on-quit (toggle-debug-on-quit), launch vscode, wait for it to freeze, then quit (C-g), then post the resulting stack trace? That'll tell us where it's freezing.

[kchanqvq]

Can you try enabling debug-on-quit (toggle-debug-on-quit), launch vscode, wait for it to freeze, then quit (C-g), then post the resulting stack trace? That'll tell us where it's freezing.

This only tells us it's in the process filter, and nothing more (as debugger won't show up inside process filter)...