Help requested: VNC display corruption after gdm restart

Ask questions regarding Gentoox, Gentoo and Linux in general in these forums and we'll do our best to help you!
Post Reply
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Help requested: VNC display corruption after gdm restart

Post by Krazy »

Hi,

Firstly let me say that the whole X11, VNC (server, client) thing confuses the heck out of me. :? So with that said...

Has anybody else noticed that restarting gdm causes VNC's display to corrupt?
(Yes, I know you loose the connection, but I'm talking about the image when you reconnect).

FYI, my setup: xbox v1.4, _headless_, with Home 4.0 [=>using VNC & ssh], XFCE4

After restarting gdm, clicking on the desktop causes coloured dots to appear on the background image (which is never refreshed). Also, the kvkbd (graphical keyboard) is shown only as a black rectangle.

Logging in via VNC (to XFCE4) works, but leads eventually to unusable display as images overlay each other and are not erased, leading to visual confusion. This happens with VNC ports 0 & 5800 (java web version).

Why restart 'gdm'? Well I'm experimenting with runlevels, and rc subsystem. I've commented out the lines in /etc/conf.d/local.start which startup gdm, and created the (hopefully) equivalent as /etc/init.d/x11 (see below).

Using my script, I can start and stop gdm via the console (via a ssh login), at will. The script appears to start and stop gdm correctly [well no 'gdm' process survives a shutdown]. I've added it to the default runlevel with

Code: Select all

rc-update add x11 default
... and after a fresh boot "x11" starts, and VNC access is fine.
However issuing /etc/init.d/x11 stop, then /etc/init.d/x11 start causes the problem to become apparent! :(

What am I missing?
Is the problem is that gdm is a display manager and not the X server?!?
Is there a VNC process that need to be killed, too?
I assume that my VNC protocol settings are fine, since they work OK with a freshly booted xbox.


Comments, please. And constructive criticism of the script :wink:

FYI, here is my script

Code: Select all

#!/sbin/runscript
# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

# Startup/Shutdown script for gdm (X11) on GentooX 
#       [Xbox port of Gentoo by ShALLaX -- http://gentoox.shallax.com]
# /etc/init.d/x11 -- not under source control. v0.1 Krazy 050725


depend() {
        use logger clock hostname
        provide x11
}

start() {
        ebegin "Starting x11"
        start-stop-daemon --start --quiet --exec /usr/bin/gdm
        /usr/bin/pgrep -x -u 0 -P 1 gdm > /var/run/gdm.pid
        eend $?
}

stop() {
        ebegin "Stopping x11"
        start-stop-daemon --stop --quiet --pidfile /var/run/gdm.pid \
                --exec /usr/bin/gdm
        eend $?
}

v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

Firstly let me say that the whole X11, VNC (server, client) thing confuses the heck out of me.
<snipped>
Is the problem is that gdm is a display manager and not the X server?!?
Well I'm still confused, but providing more info:

Doing a ps -ejH is slightly enlightening, viz:

Code: Select all

 3815  3815  3815 ?        00:00:00   gdm
 3817  3817  3815 ?        00:00:00     gdm
 3818  3818  3815 ?        00:00:00       X
 3830  3830  3830 ?        00:00:00       gdmgreeter
 3829  3823  3823 ?        00:00:00   xvkbd
From this we can see that there's 2 gdm, one of which (the nested one) appears to have spawned the X server & gdmgreeter.
xvkbd is a separate process [started by /home/gentoox/.xinitrc].
Shutting down the old 'gdm' [via my script] takes care of all the others, so I don't think I'm leaving anything behind.

VNC functionality looks like it is provided by the X server (perhaps via module), going by the following snippet from /etc/X11/xorg.conf

Code: Select all

Section "Screen"
   Identifier  "Default Screen"
   Device      "Nv"
   Monitor     "Generic Monitor"
   Option      "usevnc"
   Option      "rfbauth"      "/home/gentoox/.vnc/passwd"
   Option      "httpdir"      "/usr/share/vnc/classes/"
   Option      "httpport"     "5800"
   Option      "desktop"      "Gentoox"
   Option      "AlwaysShared"   "on"
   Option      "IdleTimeout"    "0"
   Option      "dontdisconnect"
   DefaultDepth   24
   SubSection "Display"
      Depth    24
      Modes "640x480-pal"
   EndSubSection
EndSection
I was wondered where VNC was going for its password ;)

Of course, this still doesn't answer why things get garbled when you restart gdm. :(

A bug in the X server, perhaps? :shock:
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

Ok, problem solved. I just upgraded to gdm version gdm-2.6.0.9-r2

Emerging config was a bit of a nightmare especially /etc/x11/gdm.conf.
I generally went with new config options.

However it looks like my config needs some tweaking still - I want to put back xvkbd on login screen for example.
Could be useful someday ;)
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

Rats --- don't know what I was smoking yesterday, as the problem is back now. :(

All I did was reboot, start/stop gdm a couple of times, and change a greeter theme for a look see

I also tried a different VNC client, but the problem is still there.

This is driving me nuts :evil:
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

Ok, yet more info:

Emerging Xorg-X11 to latest release (6.8.2r2) doesn't help.

Updating /usr/lib/modules/vnc.so* to version 4.3.0.999 doesn't help either. I'm grasping at straws here, I don't even know if I can upgrade just this .so file by itself ?!?
[vnc.so was previously at version 4.3.0.104
- version number found in /var/log/Xorg.0.log; search for vnc
- vnc.so appears to be part of xf4vnc]
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

Experimenting today reveals that perhaps the problem has nothing to do with starting/stopping gdm.
It may be that there's a VNC/X session problem. Read on for a slightly better description of the issue...

Firstly, let me say that a freshly booted box and first Xlogin are absolutely fine.

However, if I login via the gdmgreeter, then right-mouse-button->Quit->Quit-current-Session, then click on "Ok", it logs me out [Correct behaviour].

Then the fun starts....
... the background slowly redraws, and the top 1/4 of the screen is divided into 4 boxes. First box is painted the background colour, or black...
... the second box is all green,
... 3rd is red,
... 4th is blue

Finally the whole screen is redrawn again, and this time the whole background is correctly updated, and the greeter login bezel is shown.
Unlike the first time, xvkbd is now drawn incorrectly (only a black rectange shown, no keys visible)

If I move the mouse around the bottom 3/4 of the screen, then r.g.b dots appear in each of these 4 boxes at the top.
Its as though the whole screen is mirrored 5 times (4 in a row at the top, then once again filling remainder of screen)
I can still login, but the screen soon becomes unusable with unerased junk.

I've also posted to the gentoox irc channel, and "dragon788" said he had seen the same problem, but didn't know the cause.

Does this issue ring any bells?

I've run out of ideas on what to try next.
Suggestions please.
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

Hmm...
Emerging Xorg-X11 to latest release (6.8.2r2) doesn't help.
This may or may not be important. Emerge output said:

... the following patches are excluded:
9990_x86_6.8.0-nvxbox-20050107.patch...
9991_x86_6.8.1.904-xbox-pci-20050207.patch...
[+ a few others]

I wonder if there needs to be some special USE flags specified before emerging?
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
vplayer
1337
Posts: 442
Joined: Thu Oct 07, 2004 11:06 pm
Location: USA

Post by vplayer »

How does it work on your Pro version. Try using vnc on your pro version thats on your xbox 1.1, see if that works fine.

If I was you I would just reinstall gentoox home again and try vnc again. All depends on what you have setup already.

hope that helps,
Box: XBOX ver 1.1
Mod: Xecutor 2.3 lite+
Hdd: WD 80GB
Dash: Evox, Ava, XBMC
Linux: Gentoox Pro 4.2
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

vplayer said:
If I was you I would just reinstall gentoox home again and try vnc again.
Yeah, I might just do that, just to get back to a standard box.

Further research today reveals:-

1) Whenever the VNC screen image is corrupt, so is the main screen.
I normally use this xbox headless, so I wasn't aware that this was a problem.

2) Forget my x11 startup/shutdown script (or indeed the appropriate lines in /etc/conf.d/start.local which I have commented out).
/etc/init.d/xdm appears to do the business (with starting and stopping).
{It has dependencies on things my script didn't}


Testing with xdm shows the situation is much improved.
-Mostly (but not 100% of the time) when you logout of xfce, things will work correctly
-If the display is corrupt (say 1/7 times, I got bored of testing more than that), then /etc/init.d/xdm stop, followed by a start fixed things up.

This is a much better situation that I was facing before. At least there is now a workaround, whereas a reboot was required to fix things up before.

Of course, I though I had things fixed by updating gdm before, so who can say if this is a permanent fix. :)
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Krazy
Adept
Posts: 54
Joined: Wed Sep 10, 2003 8:41 pm
Location: Auckland, New Zealand

Post by Krazy »

I just reinstalled Home 4.0. So I now have the original gdm [2.6.0.7], and xorg-x11 [6.8.2-r1] back.

The VNC problem still exists [expected], but doesn't happen every time you quite xfce.

Also the workaround (ie /etc/init.d/xdm restart) works too. Thankfully!!

So I'd suggest commenting out the lines which start gdm in /etc/conf.d/local.start, and adding the xdm service to the default runlevel instead (if desired).

This allows you to stop & restart at will.
v1.1 xbox, stock HD, soft-modded [uxe+ dualboot: UnleashX/GentooX], with Pro2.0
v1.4 xbox, stock HD, Xecuter3 (X3 3294), running headless with Home 6.1
Post Reply