Merge branch 'optimisation'

Conflicts:
	lib/src/map_impl.dart

Author: tomaskulich

README.md

@@ -9,7 +9,7 @@ The project is forked from
 *Persistent* data structure is an immutable structure; the main difference with standard data structures is how you 'write' to them: instead of mutating
 the old structure, you create the new, independent, (slightly) modified copy of it. Typical examples of commonly used Persistent structures are String (in Java, Javascript, Python, Ruby) or Python's Tuple or Java's BigDecimal. [(Not only)](http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey) we believe such concept could be beneficial also for other data structures such as Maps, Lists/Vectors, Sets.
 
-     var couple = new PersistentMap.fromMap({'father': 'Homer', 'mother': 'Marge'});
+     var couple = new PMap.fromMap({'father': 'Homer', 'mother': 'Marge'});
      // do not (and can not) modify couple anymore
      var withChild = couple.assoc('boy', 'Bart');
      print(couple); // {mother: Marge, father: Homer}
@@ -28,8 +28,8 @@ got a 'great' idea, that instead of just computing response length he also mutat
 Finally, they work as you'd expect. How cool is this:
 
     // deeply persist the structure of Maps and Lists
-    var a = persist({[1,2]: 'tower', [1,3]: 'water'});
-    var b = persist({[1,2]: 'tower', [1,3]: 'water'});
+    PMap a = persist({[1,2]: 'tower', [1,3]: 'water'});
+    PMap b = persist({[1,2]: 'tower', [1,3]: 'water'});
     assert(a==b); 
     // kids, don't try this with standard List, it ain't going to work
     print(a[persist([1, 2])]); // prints 'tower'
@@ -48,21 +48,24 @@ One possible workaround would be to JSONize entries to string and use such strin
 Fast copying or equality done right are nice features, but this is not the only selling point here. Having different ways how to copy (shallow, deep) objects or how to compare them (== vs. equals in Java) introduces new complexity. Even if you get used to it, it still takes some part of your mental capabilities and can lead to errors.
 
 ### Structure sharing 
-    var map1 = persist({'a': 'something', 'b': bigMap});
-    var map2 = a.assoc('a', 'something completely different');
+    PMap map1 = persist({'a': 'something', 'b': bigMap});
+    PMap map2 = a.assoc('a', 'something completely different');
 Suppose you are interested, whether map1['b'] == map2['b']. Thanks to structure sharing, this is O(1) operation, which means it is amazingly fast - no need to traverse through the bigMap. Although it may sound unimportant at the first glance, it is what really enables fast caching of complex functions. Also, this is the reason, why [Om](https://github.com/swannodette/om/) framework is MUCH faster than Facebooks [React](http://facebook.github.io/react/).
 
 ## And what is the prize for this all
-Short version: size and speed. Although structure sharing makes the whole thing much more effective than naive copy-it-all approach, Persistents still are slower and bigger than their mutable counterparts. Following numbers illustrate, how much less efficient (in terms of speed or memory consumption) are Persistent data structures when benchmarking either on DartVM or Dart2JS on Node:
+In the 2.0 release, we optimized memory consumption such that the only penalty for using Persistent
+comes at lower speed. Although structure sharing makes the whole thing much more effective than naive
+copy-it-all approach, Persistents are still slower than their mutable counterparts (note however, that on
+the other hand, some operations runs significantly faster, so its hard to say something conclusive
+here). Following numbers illustrate, how much slow are Persistent data structures when benchmarking either on DartVM
+or Dart2JS on Node (the numbers are quite independent of the structure size):
 
-* DartVM memory: 2.5
-* Dart2JS memory: 6
 * DartVM read speed: 2
-* DartVM write speed: 2.5
-* Dart2JS read speed: 2
-* Dart2JS write speed: 4
+* DartVM write speed: 12 (5 by using Transients)
+* Dart2JS read speed: 3
+* Dart2JS write speed: 14 (6 by using Transients)
 
-The good part of the story is, that these numbers are not getting worse, as the Map grows - you get such performance even for Maps with tens of megabytes of data stored within them.
+Although the factors are quite big, the whole operation is still very fast and it probably won't be THE bottleneck which would slow down your app. 
 
 Some [advanced topics](https://github.com/vacuumlabs/persistent/wiki/Advanced-topics).
 

benchmark/map_memory/makefile

@@ -38,13 +38,13 @@ run: build/map_memory.js
 
 	@echo "$$INTRO"
 
-	@echo -n "DartVM persistent: "
-	@dart --old_gen_heap_size=1024 map_memory.dart ${template_size} persistent > build/temp.out 2>/dev/null
-	@tail -1 build/temp.out
-
-	@echo -n "DartVM map: "
-	@dart --old_gen_heap_size=1024 map_memory.dart ${template_size} map > build/temp.out 2>/dev/null
-	@tail -1 build/temp.out
+#	@echo -n "DartVM persistent: "
+#	@dart --old_gen_heap_size=1024 map_memory.dart ${template_size} persistent > build/temp.out 2>/dev/null
+#	@tail -1 build/temp.out
+#
+#	@echo -n "DartVM map: "
+#	@dart --old_gen_heap_size=1024 map_memory.dart ${template_size} map > build/temp.out 2>/dev/null
+#	@tail -1 build/temp.out
 
 	@echo -n "NodeJS persistent: "
 	@ (! node --max-old-space-size=1024 build/map_memory.js ${template_size} persistent > build/temp.out) 2>/dev/null
@@ -54,5 +54,9 @@ run: build/map_memory.js
 	@ (! node --max-old-space-size=1024 build/map_memory.js ${template_size} map > build/temp.out) 2>/dev/null
 	@tail -1 build/temp.out
 
+	@echo -n "NodeJS hashmap: "
+	@ (! node --max-old-space-size=1024 build/map_memory.js ${template_size} hashmap > build/temp.out) 2>/dev/null
+	@tail -1 build/temp.out
+
 	rm -f build/temp.out
 

benchmark/map_memory/map_memory.dart

@@ -7,6 +7,7 @@
 library map_memory;
 
 import 'package:vacuum_persistent/persistent.dart';
+import 'dart:collection';
 
 Map template = {};
 
@@ -15,15 +16,16 @@ List data = [];
 
 var creators = {
 
-  "persistent": () => new PersistentMap.fromMap(template),
+  "persistent": () => new PMap.fromMap(template),
 
   "transient": (){
-    var res = new TransientMap();
+    var res = new TMap();
     template.forEach((k, v) => res.doAssoc(k, v));
     return res;
   },
 
   "map": () => new Map.from(template),
+  "hashmap": () => new HashMap.from(template)
 };
 
 void run(int template_size, String mode) {

benchmark/map_speed/benchmarks.dart

@@ -8,66 +8,63 @@ part of map_bench;
 
 class WriteBenchmark extends BenchmarkBase{
 
-  final int size;
-  final BenchmarkInterface object;
+  final Map<num, num> sample;
+  BenchmarkInterface object;
+  final dynamic factory;
 
+  void setup(){
+    object = factory();
+  }
 
-  WriteBenchmark(size, object, name):
-    size = size,
-    object = object,
-    super("Writing $name($size)");
-
+  WriteBenchmark(this.sample, this.factory):super('Writing');
 
   void run(){
-
-    object.create();
-
-    for (int i = 0; i < size; i++) {
-      object.assoc("key$i", "foo");
-      object.assoc("key$i", "bar");
-    }
-
-    object.save();
-    for (int i = 0; i < size; i++) {
-      object.delete("key$i");
-    }
-
-    object.restore();
-    for (int i = 0; i < size; i++) {
-      object.delete("key$i");
+    for (var size in this.sample.keys) {
+      for (var j=0; j<this.sample[size]; j++){
+        object = factory();
+        for (var val in ["foo", "bar", "baz", "woo", "hoo", "goo", "wat"]){
+          for (int i = 0; i < size; i++) {
+            object.assoc(i*i, val);
+          }
+        }
+        for (int i = 0; i < size; i++) {
+          object.delete(i*i);
+        }
+      }
     }
   }
 }
 
 
 class ReadBenchmark extends BenchmarkBase{
 
-  final int size;
-  final BenchmarkInterface object;
-
+  final Map<num, num> sample;
+  Map<num, List<BenchmarkInterface>> objects = new Map();
+  final dynamic factory;
 
-  ReadBenchmark(size, object, name):
-    size = size,
-    object = object,
-    super("Reading $name($size)");
+  ReadBenchmark(this.sample, this.factory):super('Reading');
 
   void setup(){
-
-    object.create();
-
-    for (int i = 0; i < size; i++) {
-      object.assoc("key$i", "foo");
-    }
+    this.sample.forEach((size, count){
+      objects[size] = [];
+      for (int j=0; j<count; j++){
+        BenchmarkInterface object = factory();
+        objects[size].add(object);
+        for (int i = 0; i < size; i++) {
+          object.assoc(i*i, "foo");
+        }
+      }
+    });
   }
 
   void run(){
-
-    for (int i = size * 2; i >= 0; i--) {
-      object.get("key$i");
-    }
-
-    for (int i = 0; i <= size * 2; i++) {
-      object.get("key$i");
+    for (var size in this.sample.keys) {
+      for (var j=0; j<this.sample[size]; j++){
+        BenchmarkInterface object = objects[size][j];
+        for (int i = size; i >= 0; i--) {
+          object.get(i*i);
+        }
+      }
     }
   }
 }

benchmark/map_speed/interface.dart

@@ -8,7 +8,6 @@ part of map_bench;
 
 abstract class BenchmarkInterface<K, V>{
 
-  void create();
   void assoc(K key, V value);
   void get(K key);
   void delete(K key);

benchmark/map_speed/interface_impl.dart

@@ -7,9 +7,11 @@
 part of map_bench;
 
 class PersistentMapInterface<K, V>
-  extends EncapsulatingInterface<K, V, PersistentMap<K, V>>{
+  extends EncapsulatingInterface<K, V, PMap<K, V>>{
 
-  create() => object = new PersistentMap<K, V>();
+  PersistentMapInterface(){
+    this.object = new PMap<K, V>();
+  }
 
   assoc(K key, V value) =>
     object = object.assoc(key, value);
@@ -23,9 +25,11 @@ class PersistentMapInterface<K, V>
 
 
 class TransientMapInterface<K, V>
-  extends EncapsulatingInterface<K, V, TransientMap<K, V>>{
+  extends EncapsulatingInterface<K, V, TMap<K, V>>{
 
-  create() => object = new PersistentMap<K, V>().asTransient();
+  TransientMapInterface(){
+    this.object = new PMap<K, V>().asTransient();
+  }
 
   assoc(K key, V value) =>
     object.doAssoc(key, value);
@@ -45,7 +49,9 @@ class TransientMapInterface<K, V>
 class StandardMapInterface<K, V>
   extends EncapsulatingInterface<K, V, Map<K, V>>{
 
-  create() => object = new Map<K, V>();
+  StandardMapInterface(){
+    object = new Map<K, V>();
+  }
 
   assoc(K key, V value) =>
     object[key] = value;
@@ -91,7 +97,7 @@ class LinkedListInterface<K, V>
     while (it.isCons) {
       Cons<Pair<K, V>> cons = it.asCons;
       Pair<K, V> elem = cons.elem;
-      if (elem.first == key) {
+      if (elem.fst == key) {
         builder.add(new Pair<K, V>(key, value));
         return builder.build(cons.tail);
       }
@@ -107,7 +113,7 @@ class LinkedListInterface<K, V>
     while (it.isCons) {
       Cons<Pair<K, V>> cons = it.asCons;
       Pair<K, V> elem = cons.elem;
-      if (elem.first == key) return;
+      if (elem.fst == key) return;
       it = cons.tail;
     }
   }

benchmark/map_speed/map_speed.dart

@@ -8,31 +8,62 @@ library map_bench;
 
 import 'package:vacuum_persistent/persistent.dart';
 import 'package:benchmark_harness/benchmark_harness.dart';
+import 'dart:math';
 
 part 'benchmarks.dart';
 part 'interface.dart';
 part 'interface_impl.dart';
 
-var interfaces = {
-  //"LinkedList": () => new LinkedListInterface(),
+Map interfaces = {
   "PersistentMap": () => new PersistentMapInterface(),
   "TransientMap": () => new TransientMapInterface(),
-  "StandartMap": () => new StandardMapInterface(),
-  //"CopyMap": () => new CopyMapInterface(),
+  "Map": () => new StandardMapInterface(),
 };
 
-void main() {
-
-  for (int n in [1,10,100,1000,10000]) {
-    for (String name in interfaces.keys){
-      new ReadBenchmark(n, interfaces[name](), name).report();
-    }
-  }
+int times = 10;
 
-  for (int n in [1,10,100,1000,10000]) {
-    for (String name in interfaces.keys){
-      new WriteBenchmark(n, interfaces[name](), name).report();
+void main() {
+  var config = [
+   {'name': 'Write',
+    'creator': ((sample, factory) => (new WriteBenchmark(sample, factory))),
+//    'sizes': [{10000: 1}, ],
+    'sizes': [{500:60, 1000: 30, 1500: 20, 3000: 10}],
+   },
+   {
+     'name': 'Read',
+     'creator': ((sample, factory) => (new ReadBenchmark(sample, factory))),
+//     'sizes': [{10000: 1}, ],
+     'sizes': [{500:60, 1000: 30, 1500: 20, 3000: 10}],
+   }
+  ];
+  var result = {};
+  config.forEach((conf){
+    String mode = conf['name'];
+    var creator = conf['creator'];
+    for (Map sample in conf['sizes']) {
+      var res = {};
+      var dev = {};
+      interfaces.forEach((k,v){
+        res[k] = 0;
+        dev[k] = 0;
+      });
+      for (int i=0; i<times; i++){
+        for (String name in interfaces.keys) {
+          var meas = creator(sample, interfaces[name]).measure();
+          res[name] += meas;
+          dev[name] += meas*meas;
+        }
+      }
+      for (String name in interfaces.keys) {
+        res[name] /= times;
+        dev[name] /= times;
+        dev[name] = sqrt(dev[name] - res[name] * res[name]);
+      }
+      for (String name in interfaces.keys) {
+        var _dev = 2*(res[name]*dev['Map']+res['Map']*dev[name])/res['Map']/res['Map']/sqrt(times);
+        print('${mode} ${name} sample ${sample}: ${res[name]/res['Map']} '+
+              '+- ${_dev} (${res[name]} us)');
+      }
     }
-  }
-
+  });
 }

benchmark/mori_speed/.gitignore

@@ -0,0 +1,3 @@
+node_modules
+compiled.js
+!mori_speed.js

benchmark/mori_speed/makefile

@@ -0,0 +1,15 @@
+all: clean run
+    
+init:
+	npm up
+
+compiled.js: mori_speed.js
+	node_modules/closurecompiler/bin/ccjs mori_speed.js > compiled.js
+
+run: compiled.js
+	node compiled.js
+
+clean:
+	rm -f compiled.js
+
+