Linux Support

[sp0rus]

It seems to me that Linux support would help in getting more involvement from the open source community. Since swt-cocoa is being used in the mac version, it isn't cross-platform compatible with Linux unless I'm missing something. If it is possible to run Overview on Linux, please let me know what is needed to do so.

[jstray]

It should actually be fairly straightforward, just a matter of adding the
linux version of swt.jar into the repo, then altering the file path in
overview.sh. Would you like to try it?

• Jonathan

On Thu, Mar 1, 2012 at 6:37 PM, John Stitt <
reply@reply.github.com

wrote:

It seems to me that Linux support would help in getting more involvement
from the open source community. Since swt-cocoa is being used in the mac
version, it isn't cross-platform compatible with Linux unless I'm missing
something. If it is possible to run Overview on Linux, please let me know
what is needed to do so.

---

Reply to this email directly or view it on GitHub:
#1

[sp0rus]

I'll give it a shot this weekend if you want to assign it to me. I can get started tomorrow night.

[znmeb]

I'm close to having it work on openSUSE 12.1 - see https://github.com/znmeb/Computational-Journalism-Server/tree/master/Overview

Rough outline

1. Add an openSUSE Build Service repository and install "rbenv" and "ruby-build" packages.
2. Add "rbenv init" to "~/.bashrc"
3. Install Ruby 1.9.3 via rbenv. There's a bug in ruby-build so you won't get the latest Ruby 1.9.3 but you do get a fairly recent one.
4. Install the "eclipse-swt" package.
5. Copy the swt.jar file

The Ruby stuff seems to be working but there's some kind of CLASSPATH problem even with the right swt.jar in the right place. I have to shut down for the night but I'll be back to this tomorrow.

[znmeb]

I think I know what's happening. The Windows and Mac versions are using two jars - DJNativeSwing-SWT.jar and DJNativeSwing.jar. These are referencing binaries (shared libraries) I don't have. Here's what happens:

pushd overview-sample-files
/xhome/Projects/Computational-Journalism-Server/Overview/overview-sample-files /xhome/Projects/Computational-Journalism-Server/Overview
../overview-prototype/preprocess.sh nj-senator-menendez
Finding common bigrams...
loaded 174 stopwords.
1530 documents read
31437 unigrams, 258429 bigrams found.
6604 bigrams were kept.
Converting documents to vector space representation...
Loaded 6604 bigrams
loaded 174 stopwords.
1530 documents read
1530 uniquely keyed documents read
36320 input terms
14726 terms not culled
1530 documents with non-empty text
Creating vector and feature name files...
1530 documents encountered
1530 documents written
14726 terms
Cleaning up...
../overview-prototype/overview.sh nj-senator-menendez
Opened log file overview-log.csv
NativeSwing[1]: Exception in thread "main" java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
NativeSwing[1]: no swt-gtk-3730 in java.library.path
NativeSwing[1]: no swt-gtk in java.library.path
NativeSwing[1]: Can't load library: /home/znmeb/.swt/lib/linux/amd64/libswt-gtk-3730.so
NativeSwing[1]: Can't load library: /home/znmeb/.swt/lib/linux/amd64/libswt-gtk.so
NativeSwing[1]:
NativeSwing[1]: at org.eclipse.swt.internal.Library.loadLibrary(Library.java:284)
NativeSwing[1]: at org.eclipse.swt.internal.Library.loadLibrary(Library.java:193)
NativeSwing[1]: at org.eclipse.swt.internal.C.(C.java:21)
NativeSwing[1]: at org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:63)
NativeSwing[1]: at org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:54)
NativeSwing[1]: at org.eclipse.swt.widgets.Display.(Display.java:132)
NativeSwing[1]: at java.lang.Class.forName0(Native Method)
NativeSwing[1]: at java.lang.Class.forName(Class.java:186)
NativeSwing[1]: at org.eclipse.swt.graphics.Device.(Device.java:105)
NativeSwing[1]: at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.runNativeSide(SWTNativeInterface.java:1207)
NativeSwing[1]: at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.main_(SWTNativeInterface.java:1398)
NativeSwing[1]: at chrriis.dj.nativeswing.swtimpl.NativeInterface.main(NativeInterface.java:220)
NativeSwing[2]: Exception in thread "main" java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
NativeSwing[2]: no swt-gtk-3730 in java.library.path
NativeSwing[2]: no swt-gtk in java.library.path
NativeSwing[2]: Can't load library: /home/znmeb/.swt/lib/linux/amd64/libswt-gtk-3730.so
NativeSwing[2]: Can't load library: /home/znmeb/.swt/lib/linux/amd64/libswt-gtk.so
NativeSwing[2]:
NativeSwing[2]: at org.eclipse.swt.internal.Library.loadLibrary(Library.java:284)
NativeSwing[2]: at org.eclipse.swt.internal.Library.loadLibrary(Library.java:193)
NativeSwing[2]: at org.eclipse.swt.internal.C.(C.java:21)
NativeSwing[2]: at org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:63)
NativeSwing[2]: at org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:54)
NativeSwing[2]: at org.eclipse.swt.widgets.Display.(Display.java:132)
NativeSwing[2]: at java.lang.Class.forName0(Native Method)
NativeSwing[2]: at java.lang.Class.forName(Class.java:186)
NativeSwing[2]: at org.eclipse.swt.graphics.Device.(Device.java:105)
NativeSwing[2]: at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.runNativeSide(SWTNativeInterface.java:1207)
NativeSwing[2]: at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.main_(SWTNativeInterface.java:1398)
NativeSwing[2]: at chrriis.dj.nativeswing.swtimpl.NativeInterface.main(NativeInterface.java:220)
NativeSwing[3]: Exception in thread "main" java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
NativeSwing[3]: no swt-gtk-3730 in java.library.path
NativeSwing[3]: no swt-gtk in java.library.path
NativeSwing[3]: Can't load library: /home/znmeb/.swt/lib/linux/amd64/libswt-gtk-3730.so
NativeSwing[3]: Can't load library: /home/znmeb/.swt/lib/linux/amd64/libswt-gtk.so
NativeSwing[3]:
NativeSwing[3]: at org.eclipse.swt.internal.Library.loadLibrary(Library.java:284)
NativeSwing[3]: at org.eclipse.swt.internal.Library.loadLibrary(Library.java:193)
NativeSwing[3]: at org.eclipse.swt.internal.C.(C.java:21)
NativeSwing[3]: at org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:63)
NativeSwing[3]: at org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:54)
NativeSwing[3]: at org.eclipse.swt.widgets.Display.(Display.java:132)
NativeSwing[3]: at java.lang.Class.forName0(Native Method)
NativeSwing[3]: at java.lang.Class.forName(Class.java:186)
NativeSwing[3]: at org.eclipse.swt.graphics.Device.(Device.java:105)
NativeSwing[3]: at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.runNativeSide(SWTNativeInterface.java:1207)
NativeSwing[3]: at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.main_(SWTNativeInterface.java:1398)
NativeSwing[3]: at chrriis.dj.nativeswing.swtimpl.NativeInterface.main(NativeInterface.java:220)
Exception in thread "main" java.lang.RuntimeException: java.net.SocketException: Connection reset
at chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface.openChannel(OutProcessSocketsMessagingInterface.java:67)
at chrriis.dj.nativeswing.swtimpl.core.MessagingInterface.initialize(MessagingInterface.java:57)
at chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface.(OutProcessSocketsMessagingInterface.java:32)
at chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface$SwingOutProcessSocketsMessagingInterface.(OutProcessSocketsMessagingInterface.java:162)
at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.createOutProcessMessagingInterface(SWTNativeInterface.java:1022)
at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.createOutProcessCommunicationChannel(SWTNativeInterface.java:762)
at chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.open_(SWTNativeInterface.java:337)
at chrriis.dj.nativeswing.swtimpl.NativeInterface.open(NativeInterface.java:100)
at snappy.ui.mainHolder.main(mainHolder.java:22)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:185)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2282)
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2295)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2766)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:797)
at java.io.ObjectInputStream.(ObjectInputStream.java:297)
at chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface.openChannel(OutProcessSocketsMessagingInterface.java:65)
... 8 more
popd
/xhome/Projects/Computational-Journalism-Server/Overview
znmeb@AlgoCompSynth:/xhome/Projects/Computational-Journalism-Server/Overview>

I'm not too worried about this. I have a Windows machine I can run on for the moment. The Ruby part appears to be working fine and I can at least explore the rest of it.

[jstray]

Hmm, looks like it might actually be SWT that is missing a library. From
the looks of this your SWT.jar depends on a GTK library. But I honestly
couldn't say for sure because I've never tried it on Linux... someone else
has gotten it going though, said it was just a matter of installing the
right SWT.

• Jonathan

On Tue, May 15, 2012 at 3:30 AM, M. Edward (Ed) Borasky <
reply@reply.github.com

wrote:

I think I know what's happening. The Windows and Mac versions are using
two jars - DJNativeSwing-SWT.jar and DJNativeSwing.jar. These are
referencing binaries (shared libraries) I don't have. Here's what happens:

pushd overview-sample-files
/xhome/Projects/Computational-Journalism-Server/Overview/overview-sample-files
/xhome/Projects/Computational-Journalism-Server/Overview
../overview-prototype/preprocess.sh nj-senator-menendez
Finding common bigrams...
loaded 174 stopwords.
1530 documents read
31437 unigrams, 258429 bigrams found.
6604 bigrams were kept.
Converting documents to vector space representation...
Loaded 6604 bigrams
loaded 174 stopwords.
1530 documents read
1530 uniquely keyed documents read
36320 input terms
14726 terms not culled
1530 documents with non-empty text
Creating vector and feature name files...
1530 documents encountered
1530 documents written
14726 terms
Cleaning up...
../overview-prototype/overview.sh nj-senator-menendez
Opened log file overview-log.csv
NativeSwing[1]: Exception in thread "main" java.lang.UnsatisfiedLinkError:
Could not load SWT library. Reasons:
NativeSwing[1]: no swt-gtk-3730 in java.library.path
NativeSwing[1]: no swt-gtk in java.library.path
NativeSwing[1]: Can't load library:
/home/znmeb/.swt/lib/linux/amd64/libswt-gtk-3730.so
NativeSwing[1]: Can't load library:
/home/znmeb/.swt/lib/linux/amd64/libswt-gtk.so
NativeSwing[1]:
NativeSwing[1]: at
org.eclipse.swt.internal.Library.loadLibrary(Library.java:284)
NativeSwing[1]: at
org.eclipse.swt.internal.Library.loadLibrary(Library.java:193)
NativeSwing[1]: at org.eclipse.swt.internal.C.(C.java:21)
NativeSwing[1]: at
org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:63)
NativeSwing[1]: at
org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:54)
NativeSwing[1]: at
org.eclipse.swt.widgets.Display.(Display.java:132)
NativeSwing[1]: at java.lang.Class.forName0(Native Method)
NativeSwing[1]: at java.lang.Class.forName(Class.java:186)
NativeSwing[1]: at
org.eclipse.swt.graphics.Device.(Device.java:105)
NativeSwing[1]: at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.runNativeSide(SWTNativeInterface.java:1207)
NativeSwing[1]: at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.main_(SWTNativeInterface.java:1398)
NativeSwing[1]: at
chrriis.dj.nativeswing.swtimpl.NativeInterface.main(NativeInterface.java:220)
NativeSwing[2]: Exception in thread "main" java.lang.UnsatisfiedLinkError:
Could not load SWT library. Reasons:
NativeSwing[2]: no swt-gtk-3730 in java.library.path
NativeSwing[2]: no swt-gtk in java.library.path
NativeSwing[2]: Can't load library:
/home/znmeb/.swt/lib/linux/amd64/libswt-gtk-3730.so
NativeSwing[2]: Can't load library:
/home/znmeb/.swt/lib/linux/amd64/libswt-gtk.so
NativeSwing[2]:
NativeSwing[2]: at
org.eclipse.swt.internal.Library.loadLibrary(Library.java:284)
NativeSwing[2]: at
org.eclipse.swt.internal.Library.loadLibrary(Library.java:193)
NativeSwing[2]: at org.eclipse.swt.internal.C.(C.java:21)
NativeSwing[2]: at
org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:63)
NativeSwing[2]: at
org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:54)
NativeSwing[2]: at
org.eclipse.swt.widgets.Display.(Display.java:132)
NativeSwing[2]: at java.lang.Class.forName0(Native Method)
NativeSwing[2]: at java.lang.Class.forName(Class.java:186)
NativeSwing[2]: at
org.eclipse.swt.graphics.Device.(Device.java:105)
NativeSwing[2]: at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.runNativeSide(SWTNativeInterface.java:1207)
NativeSwing[2]: at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.main_(SWTNativeInterface.java:1398)
NativeSwing[2]: at
chrriis.dj.nativeswing.swtimpl.NativeInterface.main(NativeInterface.java:220)
NativeSwing[3]: Exception in thread "main" java.lang.UnsatisfiedLinkError:
Could not load SWT library. Reasons:
NativeSwing[3]: no swt-gtk-3730 in java.library.path
NativeSwing[3]: no swt-gtk in java.library.path
NativeSwing[3]: Can't load library:
/home/znmeb/.swt/lib/linux/amd64/libswt-gtk-3730.so
NativeSwing[3]: Can't load library:
/home/znmeb/.swt/lib/linux/amd64/libswt-gtk.so
NativeSwing[3]:
NativeSwing[3]: at
org.eclipse.swt.internal.Library.loadLibrary(Library.java:284)
NativeSwing[3]: at
org.eclipse.swt.internal.Library.loadLibrary(Library.java:193)
NativeSwing[3]: at org.eclipse.swt.internal.C.(C.java:21)
NativeSwing[3]: at
org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:63)
NativeSwing[3]: at
org.eclipse.swt.internal.Converter.wcsToMbcs(Converter.java:54)
NativeSwing[3]: at
org.eclipse.swt.widgets.Display.(Display.java:132)
NativeSwing[3]: at java.lang.Class.forName0(Native Method)
NativeSwing[3]: at java.lang.Class.forName(Class.java:186)
NativeSwing[3]: at
org.eclipse.swt.graphics.Device.(Device.java:105)
NativeSwing[3]: at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.runNativeSide(SWTNativeInterface.java:1207)
NativeSwing[3]: at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.main_(SWTNativeInterface.java:1398)
NativeSwing[3]: at
chrriis.dj.nativeswing.swtimpl.NativeInterface.main(NativeInterface.java:220)
Exception in thread "main" java.lang.RuntimeException:
java.net.SocketException: Connection reset
at
chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface.openChannel(OutProcessSocketsMessagingInterface.java:67)
at
chrriis.dj.nativeswing.swtimpl.core.MessagingInterface.initialize(MessagingInterface.java:57)
at
chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface.(OutProcessSocketsMessagingInterface.java:32)
at
chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface$SwingOutProcessSocketsMessagingInterface.(OutProcessSocketsMessagingInterface.java:162)
at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.createOutProcessMessagingInterface(SWTNativeInterface.java:1022)
at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface$OutProcess.createOutProcessCommunicationChannel(SWTNativeInterface.java:762)
at
chrriis.dj.nativeswing.swtimpl.core.SWTNativeInterface.open_(SWTNativeInterface.java:337)
at
chrriis.dj.nativeswing.swtimpl.NativeInterface.open(NativeInterface.java:100)
at snappy.ui.mainHolder.main(mainHolder.java:22)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:185)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at
java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2282)
at
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2295)
at
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2766)
at
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:797)
at java.io.ObjectInputStream.(ObjectInputStream.java:297)
at
chrriis.dj.nativeswing.swtimpl.core.OutProcessSocketsMessagingInterface.openChannel(OutProcessSocketsMessagingInterface.java:65)
... 8 more
popd
/xhome/Projects/Computational-Journalism-Server/Overview
znmeb@AlgoCompSynth
:/xhome/Projects/Computational-Journalism-Server/Overview>

I'm not too worried about this. I have a Windows machine I can run on for
the moment. The Ruby part appears to be working fine and I can at least
explore the rest of it.

---

Reply to this email directly or view it on GitHub:

#1 (comment)

[znmeb]

I think it's an openSUSE problem - the SWT is out of their repositories and is part of Eclipse. Inconvenienty they don't carry Eclipse itself any more - it didn't pass their QA in time for the release. I'll try it on Fedora.

[znmeb]

It works on Fedora 17 Beta - it looks like there's a missing dependency in the openSUSE package. Just for completeness I'll try it on Ubuntu as well, even though I only use Ubuntu when directed to by someone with a budget. ;-) When the openSUSE 12.2 beta comes out next week I'll start rattling openSUSE developer cages for the computational journalism use cases. ;-)

[znmeb]

It works on Ubuntu as well. The scripts are in https://github.com/znmeb/Computational-Journalism-Server/tree/master/Overview.

openSUSE appears to be for the moment a lost cause for Eclipse out of the openSUSE repositories. I'm guessing installing Eclipse from upstream tarballs and recompiling swt.jar will work, but given that I have both Ubuntu and Fedora working I don't see the point.

[znmeb]

I ended up writing scripts for Ubuntu, Fedora and openSUSE that do essentially the same thing:

1. Install "rbenv" and "ruby-build" from Github
2. Install a local Ruby 1.9.3 compiled from the Ruby source.
3. Download swt-gtk 64-bit binaries from the Eclipse mirrors.

Overview now works on all three, and the scripts are in the latest release of Computational Journalism Server.

By the way, I just discovered the writeup of the DiscoTree algorithm today. This looks like a major breakthrough to me - I hope you're submitting it to ACM (or IEEE if you prefer). But after reading the paper I'm curious why you're displaying the MDS results in the prototype, since the tree view is vastly superior.