alex-gutev/cl-form-types

Library for determining the types of Common Lisp forms based on information stored in the environment.

cl-form-types's Issues

custom-form-type for cl:aref can misbehave due to non-simple form-type of `the` form

cl-form-types/src/cl-functions.lisp

Lines 141 to 145 in 7a68e67

	(defmethod custom-form-type ((first (eql 'aref)) form env)
	(let ((element-type (introspect-environment:typexpand (form-type form env))))
	(if (listp element-type)
	(values (second element-type) t)
	(values t nil))))

Multiple aspects, let's start with the example that triggered this:

CL-USER> (cl-form-types:form-type `(aref (the (simple-array single-float 1) a)
                                         (the (unsigned-byte 32 i)))
                                  nil)
(VALUES T &OPTIONAL)

1. It should be (form-type (first form) env) instead of (form-type form env), and form isn't a form but rest of the form arguments.
2. One needs to additionally check if the first element of the returned type (the value of element-type) is a list with its first element cl:array or cl:simple-array or such.
3. Applying fix 1, we find that element-type gets the value (and (simple-array single-float) t). To keep things simple, if type1 is a subtype of type2, and combination can be avoided. I'm unsure if you want to apply this (and similar!) fix in combine-values-types or in the special-form-type specializing on cl:the. EDIT: Another factor is the form-type of (the type-1 form-2) should not result in type-1, because CLHS allows implementations to allow form-2 to be of a type other than type-2.

PS: Would it be recommendable to use spaces instead of tabs for indentation?

Goals of cl-form-types

I am beginning to wonder if we might have underestimated the problem of type checking. Or if I'm misunderstanding the goals of cl-form-types.

In particular, consider cl:the and declare type:

CLHS writes about the:

the specifies that the values[1a] returned by form are of the types specified by value-type. The consequences are undefined if any result is not of the declared type.

As such, in my understanding, cl:the is effectively useless for purposes of type determination or checking. While SBCL checks for types specified using the operator, at least a few other implementations like CCL do not. The run time type of (the foo-type bar-form) is solely determined by bar-form on CCL.

CL-USER> (cl-form-types:form-type `(let (a)
                                     (setq a 5)
                                     (the string a))
                                  nil)
(AND STRING T)
CL-USER> (let (a)
           (setq a 5)
           (the string a))
5

Similarly, about declare type, the consequences are undefined if the run-time type of variable differs from declared type:

The interpretation of a type declaration is as follows:

1. During the execution of any reference to the declared variable within the scope of the declaration, the consequences are undefined if the value of the declared variable is not of the declared type.
2. During the execution of any setq of the declared variable within the scope of the declaration, the consequences are undefined if the newly assigned value of the declared variable is not of the declared type.
3. At the moment the scope of the declaration is entered, the consequences are undefined if the value of the declared variable is not of the declared type.

That. That sounds like declare type is effectively useless on non-SBCL-like implementations, unless it is followed by manual type checks. So, would the goals of cl-form-types be stated as merely to deduce the declared type of the form, rather than (what I understood as) run-time type of form?

Should `variable-type` work even if the type of variable is NIL?

On some implementations, cl-form-types:form-type fails because variable-information of x returns NIL as the first value (CCL) instead of :LEXICAL (SBCL):

CL-USER> (cl-form-types::special-form-type 'locally '((declare (type string x)) x) nil)
T ; on CCL; returns STRING as expected on SBCL

cl-form-types/src/form-types.lisp

Lines 555 to 580 in 7a68e67

	(defun variable-type (variable env)
	"Determine the type of a variable.
	VARIABLE is the symbol naming the variable. This may name a
	constant as well.
	ENV is the environment in which the variable is found."
	(flet ((get-vtype (decl)
	(aif (assoc 'type decl)
	(cdr it)
	t)))
	(multiple-value-bind (type local decl)
	(variable-information variable (when *use-local-declared-types* env))
	(declare (ignore local))
	(case type
	(:constant
	`(eql ,(eval variable)))
	((:lexical :special)
	(get-vtype decl))
	(otherwise t)))))

One way to get the desired behavior could be to avoid checking for :lexical :special, but I'm unsure if that leads to other issues, does it?

---

Another issue is, while the SBCL behavior is more preferable, I'm unsure if the first return value should be :LEXICAL (it should rather be NIL perhaps? Am I missing something?):

CL-USER> (cl-environments:variable-information
          'x
          (cl-environments:augment-environment 
           nil
           :declare
           '((type string x))))
:LEXICAL
T
((TYPE . STRING))

Better handling `the` forms

This was originally suggested by @commander-trashdin: (cl-form-types:form-type '(the number 5) nil) should result in (and number (eql 5)) rather than the simpler number; that way, no information is lost.

Error after updating quicklisp/ultralisp

sbcl 2.3.10 on Windows 11

[package cl-form-types.walker].............;
; caught ERROR:
;   READ error during COMPILE-FILE:
;
;     Symbol "TRULY-DYNAMIC-EXTENT" not found in the SB-INT package.
;
;       Line: 456, Column: 69, File-Position: 13897
;
;       Stream: #<SB-INT:FORM-TRACKING-STREAM for "file C:\\Users\\simend\\quicklisp\\dists\\ultralisp\\software\\alex-gutev-cl-form-types-20230614064654\\src\\walker.lisp" {100CA31213}>

Going beyong CLTL2: constant-form-value and compiler-macroexpansion

Hi

Is there an interest in going beyond CLTL2, and/or providing a form-subtypep? For instance,

CL-USER> (trivial-form-type:form-type '(+ 2 3) nil) ; uses introspect-environment:constant-form-value
(EQL 5)
T
CL-USER> (subtypep (trivial-form-type:form-type '(+ 2 3) nil) '(unsigned-byte 8))
T
T
CL-USER> (cl-form-types:form-type '(+ 2 3) nil)
(VALUES NUMBER &OPTIONAL)
CL-USER> (subtypep (cl-form-types:nth-form-type '(+ 2 3) nil 0) '(unsigned-byte 8))
NIL
T

Such functionality may not be provided in the core system for reasons of cltl2-based portability, but a subsystem could provide it when users decide to use it.

special-form inside `block` errors on CCL

An example:

CL-USER> (cl-form-types:nth-form-type
          `(block nil
             (ccl:compiler-let ()))
          nil)

Encountered an unknown special operator COMPILER-LET, with operands:
  (NIL)
   [Condition of type CL-FORM-TYPES:UNKNOWN-SPECIAL-OPERATOR]

Restarts:
 0: [RETURN-DEFAULT-TYPE] #<RESTART CL-FORM-TYPES:RETURN-DEFAULT-TYPE #x7FC330D860BD>
 1: [RETRY] Retry SLIME REPL evaluation request.
 2: [*ABORT] Return to SLIME's top level.
 3: [ABORT-BREAK] Reset this thread
 4: [ABORT] Kill this thread

Backtrace:
  0: (#<STANDARD-METHOD CL-FORM-TYPES::BLOCK-TYPE-WALK-LIST-FORM (T T T)> COMPILER-LET (NIL) NIL)
  1: (NIL #<Unknown Arguments>)
  2: ((CCL::TRACED CL-FORM-TYPES::BLOCK-TYPE-WALK-LIST-FORM) COMPILER-LET (NIL) NIL)
  3: (CL-FORM-TYPES::WALK-FORM% (COMPILER-LET ()) NIL)
  4: ((:INTERNAL ALEXANDRIA:RCURRY) (COMPILER-LET ()))
  5: (MAP NIL #<COMPILED-LEXICAL-CLOSURE (:INTERNAL ALEXANDRIA:RCURRY) #x3020033C847F> ((COMPILER-LET ())))
  6: ((:INTERNAL (CL-FORM-TYPES::WALK-LIST-FORM (T T T))) (PROGN (COMPILER-LET ())) NIL)
  7: (NIL #<Unknown Arguments>)
  8: ((:INTERNAL CL-FORM-TYPES::WALK-FORM%) (PROGN (COMPILER-LET ())) NIL)
  9: (CL-FORM-TYPES::WALK-FORM #<Compiled-function (:INTERNAL CL-FORM-TYPES::WALK CL-FORM-TYPES::BLOCK-TYPE-WALK-FORM) (Non-Global)  #x3020012DD39F> (PROGN (COMPILER-LET ())) NIL :RESULT-TYPE NIL)
 10: (CL-FORM-TYPES::EXTRACT-RETURN-FROM-TYPES NIL (PROGN (COMPILER-LET ())) NIL)
 11: (#<STANDARD-METHOD CL-FORM-TYPES::SPECIAL-FORM-TYPE ((EQL BLOCK) T T)> BLOCK (NIL (COMPILER-LET ())) NIL)
 12: (CCL::%CALL-NEXT-METHOD (NIL #<STANDARD-METHOD CL-FORM-TYPES::SPECIAL-FORM-TYPE ((EQL BLOCK) T T)> . 17559539630812))

I have a small doubt about interpreting "The macro definition must be available for use by programs that understand only the standard Common Lisp special forms." on CLHS: http://www.lispworks.com/documentation/lw71/CLHS/Body/f_macro_.htm

Is that indicating that the bug is on CCL side and not cl-form-types - or is that inconsequential and the bug is on cl-form-types?

walker should not replace sb-kernel:the* with cl:the

cl-form-types/src/walker.lisp

Lines 583 to 588 in 0e2ba27

	#+sbcl
	(defmethod walk-list-form ((operator (eql 'sb-kernel:the*)) operands env)
	(match-form operands
	((list type form)
	(with-result (result (walk-form% form env))
	`(cl:the ,type ,result)))))

Basically, sb-kernel:the* allows for additional options so that (sb-kernel:the* (fixnum :use-annotations t) 5) is valid while (cl:the (fixnum :use-annotations t) 5) is not.

EDIT: Or, again, I'm not sure if this should be handled here or within polymorphic-functions::macroexpand-all

EDIT-2: This holds:

CL-USER> (macroexpand-1 `(sb-kernel:the* (fixnum :use-annotations t) a))
(THE FIXNUM A)
T

`function` type should avoid `cl:*`

CL-USER> (cl-form-types:form-type `(lambda (x) (1+ x)) nil)
(FUNCTION NIL (FUNCTION (*) (VALUES NUMBER &OPTIONAL)))

SBCL 2.1.10 yields a warning for function type-specifiers with cl:*

; caught WARNING:
;   * is not permitted as an argument to the FUNCTION type specifier

Values types should not be combined with `NIL`

Essentially:

CL-USER> (cl-form-types:form-type '(the string (values "" 1.0)) nil)
(VALUES (AND STRING (SIMPLE-ARRAY CHARACTER (0))) (AND NIL (EQL 1.0)))

The second values-type should be just (EQL 1.0) or (AND T (EQL 1.0) rather than (AND NIL (EQL 1.0)).

`values` type needs more normalization (values inside values)

Example:

(declaim (ftype (function (*) (values &rest number)) bar))
(defun bar (l) (values-list l))
(cl-form-types:form-type '(values "" (bar x)) nil)
; => (VALUES (SIMPLE-ARRAY CHARACTER (0)) (VALUES &REST NUMBER))
; should rather just be (VALUES (SIMPLE-ARRAY CHARACTER (0)) (OR NULL NUMBER)) 
; SBCL does the addition of (OR NULL ...) for (defun baz (l) (values "" (bar l)))