[colm] alphabet is signed by default, preventing ranges that cross signedness boundary

[adrian-thurston]

[colm] alphabet is signed by default, preventing ranges that cross signedness boundary

[adrian-thurston]

Can hack the load of colm programs to break ranges that cross the 0x7f boundary. Provides a nice quick and dirty. This hack modifies all ranges, regardless of whether or not they are specified using hex.

Probably makes sense to switch to an unsigned character. Ragel is signed by default because of origins in C and pervasiveness of char type. But probably doesn't make much sense in Colm.

If we don't switch to unsigned then maybe right solution is to implement this just for ranges specified in hex. Although that does add an exception to the implementation that takes up some mental space.

diff --git a/colm/parsetree.cc b/colm/parsetree.cc
index 572f061..5791973 100644
--- a/colm/parsetree.cc
+++ b/colm/parsetree.cc
@@ -1228,16 +1228,38 @@ FsmGraph *Range::walk( Compiler *pd )
 	delete lowerFsm;
 	delete upperFsm;
 
+	bool span0 = false;
+	if ( lowKey >= 0 && highKey < 0 )
+		span0 = true;
+
 	/* Validate the range. */
-	if ( lowKey > highKey ) {
+	if ( !span0 && lowKey > highKey ) {
 		/* Recover by setting upper to lower; */
 		error(lowerLit->loc) << "lower end of range is greater then upper end" << endl;
 		highKey = lowKey;
 	}
 
 	/* Return the range now that it is validated. */
-	FsmGraph *retFsm = new FsmGraph();
-	retFsm->rangeFsm( lowKey, highKey );
+	FsmGraph *retFsm = 0;
+	if ( span0 ) {
+		/* Usual case. */
+		FsmGraph *first = new FsmGraph();
+		FsmGraph *second = new FsmGraph();
+		Key k128 = 127;
+		Key kn127 = -128;
+		first->rangeFsm( lowKey, k128 );
+		second->rangeFsm( kn127, highKey );
+
+		first->unionOp( second );
+		first->minimizePartition2();
+		retFsm = first;
+	}
+	else {
+		/* Usual case. */
+		retFsm = new FsmGraph();
+		retFsm->rangeFsm( lowKey, highKey );
+	}
+
 	return retFsm;
 }