Jump to content
Excelsior Forums


  • Content count

  • Joined

  • Last visited

Community Reputation

0 Neutral

About MemoryHungry

  • Rank
  • Birthday 01/01/1

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. Considering that this was example code provided to illustrate a problem, optimizing makes little sense. Furthermore, if the speed improvement is also seen with the Oracle JRE, then the performance difference would remain. Unfortunately, using "Very aggressive" inlining makes no difference for our code. Jet compiled code still is between 2x and 5x slower. Performance of x86 Jet code is less important for us, since using RAM beyond the x86 limits is essential for our software. That said, we find similar performance differences for x86 compiled code when using about 1 GB of RAM. In two tests, performance was 1.5x slower than JRE for one data set, and 6.5x slower for a second data set that was about 2x larger. The performance penalty alone is bad enough, but seeing that it gets worse in what looks like N-square fashion with larger data sets is a big problem, too. We will have to stop using Jet for future releases of our software.
  2. We just encountered similar performance issues with Jet 11.3. Performance on memory-intensive tasks is about 5-10x slower than the with JRE 1.8.0_121, and 2-5x slower than with Jet 9. This happens when memory use exceeds 1-1.5 GB. Even worse, we observe early OutOfMemory errors if virtual memory use is turned off. "Early" means that the errors while there is still plenty of empty memory for heap growth (e.g. after using 780 MB with > 2 GB free RAM). There are some indications that the slow performance is also due to memory management issues. Object allocations and garbage collection are very fast for smaller data sets, but GC slows down to significantly slower than JRE speeds when memory use goes above 1-1.5 GB. Apparently, the performance issues for large data sets originally reported here are still present in the latest Jet version, even though you have had a sample program illustrating this issue. Can you please provide an update on this issue?
  3. MemoryHungry

    JET Runtime: System Exception

    We have a similar problem with crashes of Jet-compiled applications. We see a Jet Runtime crash when running our JUnit tests on Windows XP: This happens when compiling with Jet 9 or 10, but not with Jet 8. The crash always happens in the same test, but apparently is the result of a lingering corruption issue: when running just the test that crashes, or a combination of this test plus several other test classes, the crash does not happen. When removing the class that crashes from the tests, the crash disappears, but we get test failures in other classes that are not observed on other test configurations (run from JRE on Windows or OS X; Jet 10 compiled on 64-bit Windows or Windows 7, 32-bit; Jet 8 compiled).
  4. We have several customers reporting nullpointer exceptions on 64-bit windows machines. The exceptions appear at different places. In one case, this is in single-thread code that has not been changed for years. We cannot reproduce the problem here, nor has any OS X or 32-bit Windows user reported similar problems. The code was compiled with Jet 8 MP 1 (before MP1, one customer reported crashes that were fixed after applying update 1). Can you please comment on whether there is any reason to believe that updating to Jet 9 might solve these issues? On your web site, you seem to strongly recommend the 32-bit version over the 63-bit version. Unfortunately, we need the 64-bit version to support large heap spaces. The Jet 9 description on your web site only mentions improvements and support for Java 7, but no bug fixes. Any help or suggestions will be appreciated.
  5. MemoryHungry

    Graphics issues in Jet 7

    After upgrading from Jet 6.4 to 7.0, we noticed multiple graphics issues in our automated JUnit tests. Menu fonts get changed after about 1 min into the test; on some test runs, there are also issues with multiple versions of each window being drawn. Problems were observed in Windows7 Pro; the problems are not existent when running the code using JRE 1.6.0_16 on the same machine. Since there are no features in version 7.0 relevant to our application, we will not have time to trouble-shoot this, and instead continue using version 6.4.
  6. MemoryHungry

    64-bit Support

    On February 14, 2008, zztop wrote: Since we need a 64-bit version for Windows, this promise played an important role in our decision to purchase Jet, as well as in our decision to renew the support agreement. Our scientific application needs to support > 4 GB for some of our customers. Now, in the post from September 22, 2009, zztop wrote: Not only was the "more definitive" promise not kept, but it is questionable if 64-bit support will be added in the next major update after 7.0. I can understand that features get delayed. But do you really feel no obligation to deliver on previously made promises?
  7. MemoryHungry

    Feature request: .ini files

    What are the chances that .ini file support makes it into version 7? This is still an important feature for us, we often encounter computer-specific problems that could most easily be solved with .ini files.
  8. MemoryHungry

    Feature request: .ini files

    I suggest that you add support for setting parameters and environment variables through .ini files that are read automatically when the application launches (if they are present). An example for this functionality are installers created by Advanced Installer (the .ini files have the same name as the application and reside in the same directory). I understand flags can be set using a .bat file to launch the application. However, this leads to two different ways of launching the application. It's fine for development use, but average users would often be confused. We found that .ini files have been very useful to work around problems discovered after product launch. Over the past years, we have encountered several problems which were easily solved by adding flags to .ini files. Two examples are graphics acceleration problems that some users experienced, and Java bugs like the user.home bug. Typically, this issues were limited to specific system configurations, and therefore only discovered after deployment. Modifying a line in a .ini file, for example to turn 2D acceleration off, was a nice workaround that enabled users to be up an running again with minimal delays. There are other uses for .ini files, too, like setting memory parameters or flags like headless, where .ini files would be an elegant solution.
  9. MemoryHungry

    Runtime.maxMemory() broken

    Which is great, because it allows us to at limit swapping to a reasonable amount. With unlimited swapping, the slowdowns can be extreme (and the users blame our application for being so slow). But since the heap will never grow larger than the physical memory, min(3GB, amount of physical memory) would be a more reasonable answer than always returning 3GB if this compiler flag is set. I saw Windows's behavior of actually using virtual memory to swap out (mostly) other processes, since it takes a while before the application actually get the entire physical memory space. But even with this limitation and other limitations discussed before, changing maxMemory to return the amount of physical memory if this compiler flag is set would be useful to detect when our application approaches the maximum heap space (we might actually warn the first time when approaching 75% or so). Most of our users are at universities and have to use old computers, so my guess is that at least 75% have 1 GB or less RAM installed.
  10. MemoryHungry

    Runtime.maxMemory() broken

    Your quote cuts off what I was talking about in a misleading way. I specifically stated "compiled to throw OutOfMemoryErrors when physical memory gets low". Your documentation indicates that virtual memory will not be used in such a situation (the two options are "use virtual memory" and "throw OutOfMemoryErrors"). Based on Jet documentation, I would expect that compiling an application with this flag will NEVER lead to a situation where the heap is larger than the actual physical memory (although I understand perfectly well that the OS will typically use virtual memory). If this is a misunderstanding, could you clarify what exactly this compiler option does?
  11. MemoryHungry

    Runtime.maxMemory() broken

    I had just composed a rather lengthy answer, when the board logged me out automatically and all that work was lost. So this one will be shorter. Bottom line: When using automatic heap sizes, Jet's implementation of maxMemory() is useless. It always returns 3 GB, even though Java on common Win32 systems does not allow anywhere close to this much. The wrong "3 GB" answer will also be returned if there is absolutely no way that the heap could ever grow so big (for example on a machine with < 1 GB RAM, compiled to throw OutOfMemoryErrors when physical memory gets low. But Excelsior does not consider this as a bug. Since our objective is to avoid application crashes and hangs, and our previous "heuristic" workaround that works perfectly fine on Sun JVMs relies on maxMemory(), we will have to add more heuristic workarounds, this time for the shortcomings of Jet. i Which means we will be tied to the current version for as long as possible, and have no incentive to extend the maintenance. So I'll just leave a warning to other users who may have memory-hungry applications: just like Sun and Apple JVMs, Jet-compiled applications can hang, misbehave, or crash when running out of memory. Since compiled Jet code seems to execute faster than normal Java, the likelihood of OutOfMemoryErrors in multiple threads (event thread, worker threads, timer threads, garbage collection, ...) very shortly after each other appears to be significantly increased. These multiple-thread OutOfMemoryErrors are the ones that can cause the most severe problems. Existing workarounds for "fixed max size" heaps that use maxMemory will not work for Jet-compiled applications that use an adaptive heap size, since the value returned by Runtime.maxMemory is fixed and useless.
  12. MemoryHungry

    Runtime.maxMemory() broken

    My issue is that maxMemory() (a) behaves differently than any Java 1.4-1.6 version on Windows and OS X, and ( returns wrong values in Jet. In Jet, maxMemory will return 3 GB, even if the application was compiled so throw OutOfMemoryErrors when "the Runtime exceeds the physical memory threshold" (cited form Jet help). When running on a machine with substantially less than 3 GB of RAM, the application will clearly NEVER try to use 3 GB of RAM. It will throw an OutOfMemoryError at about the time that heap size is equal to built-in RAM. Thus, maxMemory is clearly broken in Jet. The real issue is that JVMs tend to hang or crash when memory gets really tight. Furthermore, JVMs will not start (or crash immediately) if the -Xmx parameter specified exceeds the available memory (or, more exactly, if no virtual memory block of the requested size is available). With our scientific application that may consume memory rapidly, relying on OutOfMemoryErrors to indicate low-memory conditions was not sufficient. The problem is that the "worker" thread may consume available memory just before some other thread also needs memory - for example the event thread to update a progress indicator. OutOfMemoryErrors in other threads can lead to hangs, windows that remain open and cannot be closed, classes that cannot be loaded anymore (if the error happened in static initializer code), and crashes. In both Sun and Apple Java, we had a workaround that used Runtime's memory functions to see when our application approaches the memory limits. It relied on maxMemory, which in all current Sun and Apple implementations from 1.4 to 1.6 returns the max heap size (although specific versions require minor workarounds for small accuracy issues). Jet's simplistic approach to always return 3GB obviously breaks this. Your response is not helpful (it does not even give the impression that you are trying to be helpful). We have spent several days trying to work around this limitation in Jet, without success. In JUnit memory stress tests that work without problems on OS X and Windows with Sun JREs, we still get erratic behavior. I attached a screen shot that shows "extMalloc failed to allocate" error messages dumped to the console during one incident. One possible solution to this problem would be as follows: when compiled with the "handling low memory" flag to not use virtual memory, maxMemory() should return the amount of currently available physical memory. When using virtual memory, the amount returned should be the amount of free virtual memory. The application obviously should not attempt to use more physical respectively virtual memory than is available, since such an attempt would certainly fail. Therefore, these answers would make much more sense than always returning 3GB, as Jet does now.
  13. MemoryHungry

    Runtime.maxMemory() broken

    Our application may use a lot of memory, and needs to detect and handle low memory situations. When using the JRE, we rely on Runtime.getRuntime().maxMemory() for an estimate of the max heap space, which works well. One main reason why we purchased Jet was the option to use adaptive memory, but we ran into a problem with our existing (multi-platform) code. With Jet, maxMemory() always appears to return about 3 GB, even on a machine with substantially less memory and limited swap space (e.g. 700 MB RAM and 50 MB swap space). This is obviously a wrong answer, and leads to OutOfMemory errors. That leads to problems because they can appear in multiple threads; the worst problems occur when the errors are thrown during Thread creation and in static initializer code. Could you fix this so that maxMemory() returns a more reasonable number? Of course, an estimate based on actual available memory (including swap space) would be best. But even something like: min (3GB, built-in RAM + swap space) would give a more useful number. A related issue is knowing how much build-in RAM the computer actually has. We could also use this as a guideline for warning the user about swap slowdowns. Is there a function or DLL where we could get the amount of built-in RAM?