OWFS 2.7p4

From OWFS site: OWFS is an easy way to use the powerful 1-wire system of Dallas/Maxim.

I've found a bug in OWFS 2.7p4. A badly written lock results in access to already freed memory region. In my case it usually shows itself in 1sec to 5 minutes of working in heavily simultaneous mode with many 1-wire devices. With this bugs OWFS ends up in a crash in a random place (usually tsearch complains about damaged list). GDB was not of much help here. But Valgrind (a really great tool) helped a lot.

==16965== Thread 9:
==16965== Invalid read of size 4
==16965==    at 0x405F7A5: LockGet (ow_locks.c:142)
==16965==    by 0x4066195: FS_r_given_bus (ow_read.c:229)
==16965==    by 0x40663B2: FS_read_distribute (ow_read.c:191)
==16965==    by 0x40668AD: FS_read_postparse (ow_read.c:106)
==16965==    by 0x4066AEA: FS_read (ow_read.c:58)
==16965==    by 0x409E26D: fuse_fs_read (in /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A2A12: (within /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A5F48: (within /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A6EAF: (within /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A86D5: fuse_session_process (in /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A4AD4: (within /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40C8382: start_thread (in /lib/libpthread-2.7.so)
==16965==  Address 0x43b8dc8 is 0 bytes inside a block of size 16 free'd
==16965==    at 0x402465C: free (vg_replace_malloc.c:323)
==16965==    by 0x41B281A: tdelete (in /lib/libc-2.7.so)
==16965==    by 0x405F62B: LockRelease (ow_locks.c:164)
==16965==    by 0x40662AD: FS_r_given_bus (ow_read.c:236)
==16965==    by 0x40663B2: FS_read_distribute (ow_read.c:191)
==16965==    by 0x40668AD: FS_read_postparse (ow_read.c:106)
==16965==    by 0x4066AEA: FS_read (ow_read.c:58)
==16965==    by 0x409E26D: fuse_fs_read (in /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A2A12: (within /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A5F48: (within /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A6EAF: (within /usr/lib/libfuse.so.2.7.3)
==16965==    by 0x40A86D5: fuse_session_process (in /usr/lib/libfuse.so.2.7.3)

It has already been working for 2 hours, but I will wait a little bit more before send this patch to mailing list of OWFS.

Here is the patch.

P.S. Also Valgrind shows total memory leaks of 4kb.

(update 19 feb 2009: now it has been in work for nearly one year without any crash)

Java's Thread Priorities in Linux

As I told yesterday, Sun Java 5/6 has a "broken" priority policy for linux. There is a related bug in java's bugtracking system. So if to mention it briefly, there are two thread priority policies for linux. The first one, with code 0, specifies to ignore all thread priorities at all making all threads equal for linux scheduler. The second policy model, with code 1, maps java priorities to calls to the system setpriority function. By default java uses thread priority policy 0. You can say java to use policy 1 by invoking it with -XX:ThreadPriorityPolicy=1 command line argument. Unfortunately, this will work only if you start java with root privileges because the setpriority function sets nice level for processes and a process can't lower a nice level. This is a bad news. The good news is that Sun's java has another bug (I hope this bug will not be fixed). Look at the following code from the java 6 update 3 (build 105) sources (os_linux.cpp):

  if (ThreadPriorityPolicy == 1) {
   // Only root can raise thread priority. Don't allow ThreadPriorityPolicy=1
   // if effective uid is not root. Perhaps, a more elegant way of doing
   // this is to test CAP_SYS_NICE capability, but that will require libcap.so
   if (geteuid() != 0) {
     if (!FLAG_IS_DEFAULT(ThreadPriorityPolicy)) {
       warning("-XX:ThreadPriorityPolicy requires root privilege on Linux");
     }
     ThreadPriorityPolicy = 0;
   }
 }

It is meant to reset command line argument from 1 to 0 if the effective user is not root.

Look here:

OSReturn os::set_native_priority(Thread* thread, int newpri) {
 if ( !UseThreadPriorities || ThreadPriorityPolicy == 0 ) return OS_OK;

 int ret = setpriority(PRIO_PROCESS, thread->osthread()->thread_id(), newpri);
 return (ret == 0) ? OS_OK : OS_ERR;
}

OSReturn os::get_native_priority(const Thread* const thread, int *priority_ptr) {
 if ( !UseThreadPriorities || ThreadPriorityPolicy == 0 ) {
   *priority_ptr = java_to_os_priority[NormPriority];
   return OS_OK;
 }
...
}

I've highlighted the interesting code. Do you see how it checks for policy? It means that if we set ThreadPriorityPolicy to ... say to 666, java will still work like the policy is set to 1. And the check that resets ThreadPriorityPolicy from 1 to 0 for non-root users will not be triggered. Whee! So in fact we can use the policy 1 for non-root users running java with the command line argument -XX:ThreadPriorityPolicy=666. But stop would say you, what will it give us if the system's setpriority function fails for non-root user? It gives us enough if we can run our lovely java initially with nice level -5. A level -5 is the nice level used for HIGHEST priority threads in java as you can see from the following code snippet:

int os::java_to_os_priority[MaxPriority + 1] = {
 19,              // 0 Entry should never be used

  4,              // 1 MinPriority
  3,              // 2
  2,              // 3

  1,              // 4
  0,              // 5 NormPriority
 -1,              // 6

 -2,              // 7
 -3,              // 8
 -4,              // 9 NearMaxPriority

 -5               // 10 MaxPriority
};

Though this looks not fair and totally unholy, I think it is still better then running java with priorities ignored or under root. Thus you can run your java safely and well-scheduled with the command like this:

nice -n -5 su -c 'java -XX:ThreadPriorityPolicy=666 ThreadTester' akshaal

There is other way if you have no access to root user.

java -XX:ThreadPriorityPolicy=666 \
          -XX:JavaPriority10_To_OSPriority=0 \
          -XX:JavaPriority9_To_OSPriority=1 \
          -XX:JavaPriority8_To_OSPriority=2 \
          -XX:JavaPriority7_To_OSPriority=3 \
          -XX:JavaPriority6_To_OSPriority=4 \
          -XX:JavaPriority5_To_OSPriority=5 \
          -XX:JavaPriority4_To_OSPriority=6 \
          -XX:JavaPriority3_To_OSPriority=7 \
          -XX:JavaPriority2_To_OSPriority=8 \
          -XX:JavaPriority1_To_OSPriority=9 \
     ThreadTester

The drawbacks of this method is that your normal priority threads will run with nice level 5 (although threads with the highest level will run with nice level 0 and lowest priority thread with nice level 9). That is it. Enjoy while these bugs (or features) are not fixed yet :).

JRobin 1.5.8 Speedup

One month ago I created a patch for JRobin. And today I've submitted it to project's page on sourceforge. For graphs with CDEFs this patch speeds up graph creation by the factor of 10 and at least by the factor of 1.5 for graphs without CDEFs. In addition it makes possible to draw arbitrary graphic on top of created graph and use full-featured java code instead of CDEFs (extends Plottable class).

The patch

Java 6 & AMD64

I've noticed that Java 6 update 4 for amd64 always crashes in arbitrary places (for instance, eclipse crashes compiling a class). Whenever you need to use Java 6 Update 4 on amd64 but it keeps crashing, use -Djava.compiler=none - to disable Java6's buggy JIT. In a case your requirements are not Java 6 you can use Java 5 update 14 on amd 64 for it looks stable.

Current thread (0x00002aab2d10e800):  JavaThread "CompilerThread1" daemon [_thread_in_native, id=20462, stack(0x00000000406b2000,0x00000000407b3000)]

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000000000

Registers:
RAX=0x0000000000000000, RBX=0x00002aab311cc4d8, RCX=0x00002aab3137e9b0, RDX=0x0000000000000046
RSP=0x00000000407aee70, RBP=0x00000000407aeee0, RSI=0x00002aab3137e9b0, RDI=0x00002aab311cc4d8
R8 =0x00002aab3137ea08, R9 =0x00002aab3137e9b0, R10=0x00002aab3137e9b0, R11=0x0000000000000000
R12=0x00002aab311cc4a0, R13=0x0000000000000000, R14=0x00002aab2cd3b870, R15=0x0000000000000001
RIP=0x00002b5f70ae122a, EFL=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
  TRAPNO=0x000000000000000e

Top of Stack: (sp=0x00000000407aee70)
0x00000000407aee70:   00002aab30548b88 00000001ffffffff
0x00000000407aee80:   00002aab3137e9b0 0000000270a85331
0x00000000407aee90:   00002aab30548b80 00000007407afaa0
0x00000000407aeea0:   00000000407afb68 01000000000004a7
0x00000000407aeeb0:   00000000407afaa0 00002b5f7112a4f0
0x00000000407aeec0:   00000000407afaa0 00000000407af030
0x00000000407aeed0:   00000000407aeff0 00000000407aefc0
0x00000000407aeee0:   00000000407af100 00002b5f70ae076c
0x00000000407aeef0:   00000000407afb68 00000000407af0b0
0x00000000407aef00:   00002aab330b0fb0 00002aab30000020
0x00000000407aef10:   00000000407aef80 00002b5f711438d0
0x00000000407aef20:   00002aab307d8a20 0000000000000151
0x00000000407aef30:   00000000407afbc0 00002b5f70608bbc
0x00000000407aef40:   00000000407b2950 00000000407aef80
0x00000000407aef50:   00002b5f71146040 00002aab3100b990
0x00000000407aef60:   0000000000000151 00002b5f70f1a9d2
0x00000000407aef70:   0000000040110b90 00002b5f70f1aaa1
0x00000000407aef80:   00000000407b2950 00000000407aefc0
0x00000000407aef90:   00002b5f71146040 0000000000007fe8
0x00000000407aefa0:   00002aab307f5d10 00002b5f70f1a9d2
0x00000000407aefb0:   0000000040110b90 00002b5f70f1aaa1
0x00000000407aefc0:   00002b5f7112a4f0 00002b5f0000000e
0x00000000407aefd0:   00000000407b1450 00000000407afaa0
0x00000000407aefe0:   00000000000008c8 00002aab2d10ec70
0x00000000407aeff0:   000000000000000d 00000000407b1450
0x00000000407af000:   00002aab32c78490 00002b5f70a84d00
0x00000000407af010:   00002aab311bd4b0 00000000000004a8
0x00000000407af020:   00000000407af0b0 00002aab332443e0
0x00000000407af030:   000000000000000c 00000000407b1450
0x00000000407af040:   00002aab3178f090 00002aab31ce1aa0
0x00000000407af050:   00002aab3184eb70 00002aab3184f6b0
0x00000000407af060:   00002aab330c8c60 00000000407afbc0

Instructions: (pc=0x00002b5f70ae122a)
0x00002b5f70ae121a:   48 89 cf 41 ff 95 80 00 00 00 48 89 df 49 89 c5
0x00002b5f70ae122a:   8b 00 41 21 44 24 38 41 8b 45 04 21 43 04 41 8b
Stack: [0x00000000406b2000,0x00000000407b3000],  sp=0x00000000407aee70,  free space=1011k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x1f122a]
V  [libjvm.so+0x1f076c]
V  [libjvm.so+0x244525]
V  [libjvm.so+0x241035]
V  [libjvm.so+0x1e05c7]
V  [libjvm.so+0x248ec8]
V  [libjvm.so+0x248866]
V  [libjvm.so+0x62a3f9]
V  [libjvm.so+0x6246a1]
V  [libjvm.so+0x505eea]


Current CompileTask:
C2:581      org.eclipse.core.internal.dtree.DataTreeNode.forwardDeltaWith([Lorg/eclipse/core/internal/dtree/AbstractDataTreeNode;[Lorg/eclipse/core/internal/dtree/AbstractDataTreeNode;Lorg/eclipse/core/internal/dtree/IComparator;)[Lorg/eclipse/core/internal/dtree/AbstractDataTreeNode; (469 bytes)

Debian + Eclipse + Java + AMD64

This is for those who use debian on amd64 with java from package . If you have a problem running eclipse (exit code=13 or alike) just install sun's java and enjoy! There is a something wrong with the java package in debian (sun-java6-*). I've spent two days trying get it to work.