Is there a memory-efficient replacement of java.lang.String?

With a Little Bit of Help From the JVM… WARNING: This solution is now obsolete in newer Java SE versions. See other ad-hoc solutions further below. If you use an HotSpot JVM, since Java 6 update 21, you can use this command-line option: -XX:+UseCompressedStrings The JVM Options page reads: Use a byte[] for Strings which … Read more

Dereferencing type-punned pointer will break strict-aliasing rules

The problem occurs because you access a char-array through a double*: char data[8]; … return *(double*)data; But gcc assumes that your program will never access variables though pointers of different type. This assumption is called strict-aliasing and allows the compiler to make some optimizations: If the compiler knows that your *(double*) can in no way … Read more

Memory Allocation/Deallocation Bottleneck?

It’s significant, especially as fragmentation grows and the allocator has to hunt harder across larger heaps for the contiguous regions you request. Most performance-sensitive applications typically write their own fixed-size block allocators (eg, they ask the OS for memory 16MB at a time and then parcel it out in fixed blocks of 4kb, 16kb, etc) … Read more

Java split String performances

String.split(String) won’t create regexp if your pattern is only one character long. When splitting by single character, it will use specialized code which is pretty efficient. StringTokenizer is not much faster in this particular case. This was introduced in OpenJDK7/OracleJDK7. Here’s a bug report and a commit. I’ve made a simple benchmark here. $ java … Read more

deepcopy() is extremely slow

Actually, deepcopy is very slow. But we can use json, ujson, or cPickle. we can use json/cPickle to dump an object, and load it later. This is my test: Total time: 3.46068 s File: Function: test at line 15 Line # Hits Time Per Hit % Time Line Contents ============================================================== 15 @profile 16 def … Read more

very quickly getting total size of folder

You are at a disadvantage. Windows Explorer almost certainly uses FindFirstFile/FindNextFile to both traverse the directory structure and collect size information (through lpFindFileData) in one pass, making what is essentially a single system call per file. Python is unfortunately not your friend in this case. Thus, os.walk first calls os.listdir (which internally calls FindFirstFile/FindNextFile) any … Read more

What’s the fastest way to read from in Java?

Is there any faster way of doing this in Java? Yes. Scanner is fairly slow (at least according to my experience). If you don’t need to validate the input, I suggest you just wrap the stream in a BufferedInputStream and use something like String.split / Integer.parseInt. A small comparison: Reading 17 megabytes (4233600 numbers) using … Read more