Re: load and use a ".o" file?

[j-anthony@rcn.com (Jon S. Anthony)]

Stephen Leake <stephen_leake@acm.org> writes:

> (Jon S. Anthony) writes:
> 
> > Stephen Leake <stephen_leake@acm.org> writes:
> > 
> > >  (Jon S. Anthony) writes:
...

> > I think most people in the "dynamic community" would say you need at
> > least the following characteristics to be "truly dynamic":
> > 
> > 1. Dynamic typing.  This is, strong typing on _objects_ as
> >    differentiated from "variables/locations".
> 
> Ok. I don't think that impacts on the issue of adding new code at runtime.

Actually it does, but in non obvious ways.  For this discussion, it is
safe to ignore it.

> > 2. Complete introspection of the language's computation and type
> > model. This is directly and explicitly available at the level of the
> > programmer (not just the implementer). NOTE: This does _not_ mean or
> > imply access to the underlying _implementation_.
> > 
> >    Some things this _does_ imply:
> > 
> >    * New function objects may be added to a running system at any time
> >      and existing functions may be so modified at any time.
> 
> Hmm. The English meaning of "introspection" is "to examine oneself";
> that's not the same as creating new code. But you get to use the term
> the way you want to ...

Fair enough; while the explicit addition of "self modifying" would be
clearer for c.l.a, I believe that is taken for granted in the "dynamic
community".  It is not always easy to clearly change gears in usenet
discussions...

> >    * The ability to create new types and/or classes at runtime;
> >    modify existing classes (with complete instance migration).
> > 
> > 
> > A couple examples which fully have this would be Common Lisp and
> > Smalltalk.  Neither are "interpreter" based[1].  Most Common Lisp
> > implementations have optimizing native compilers that typically
> > produce code within about 1.5-2.5 of well crafted C code[2].  
> 
> Hmm. Do those compiled versions include the ability to create new code
> at runtime, where the new code is also compiled, not interpreted?

Yes.

> I'd be very surprised if that were true (I have not used a system
> where that is true, but I have used only a couple Lisps, and no
> Smalltalks).  If it is true, what is the linking mechanism used?

It is a pleasent surprise, I hope.  The "linking" mechanism is simply
resolution on a symbol's function slot.  This is part of the
introspective aspect of a Lisp system.  The native code is "just"
placed in a code vector.


> > CL-USER(1):
> >  (defun foo (l r)
> >    (declare (fixnum l r) (optimize (speed 3 safety 1 debug 1)))
> >    (+ l r))
> > => FOO
> > 
> > CL-USER(2): (compile 'foo)
> > => FOO
> > 
> > CL-USER(3): (disassemble 'foo) ; Just to show we are native (on Intel...)
> > ;; disassembly of #<Function FOO>
> > ;; formals: L R
> > 
> > ;; code start: #x7140a65c:
> >    0: 83 f9 02    cmpl	ecx,$2
> >    3: 74 02       jz	7
> >    5: cd 61       int	$97             ; EXCL::TRAP-ARGERR
> >    7: 80 7f 97 00 cmpb	[edi-105],$0    ; SYS::C_INTERRUPT
> >   11: 74 02       jz	15
> >   13: cd 64       int	$100            ; EXCL::TRAP-SIGNAL-HIT
> >   15: 03 c2       addl	eax,edx
> >   17: f8          clc
> >   18: 8b 75 fc    movl	esi,[ebp-4]
> >   21: c3          ret
> 
> Hmm. You don't actually demonstrate calling foo, but I'll grant you
> that :).

That was a little further below in the example which compiles and
loads foo from a file.


> But I'd still like to understand what the linking process implied by
> (compile 'foo). As far as I can see, it must do the equivalent of
> Windows DLL or Unix so.

No, this is part of the underlying introspective environment.  It is
actually much simpler than the contortions involved in DLL/so type
stuff.  It is also fine grained - a function at a time or even a
functions symbol at a time.

> Just because that mechanism is hidden from the user (as it should
> be) doesn't mean it isn't there!

Yes, it is there, and you can even get at the important user
(programmer) level bits yourself.  Here is a few interactions after
the compile and load of the junk.cl file:

CL-USER(18): (inspect 'bar)
The symbol BAR @ #x7140990f
  which is an INTERNAL symbol in the COMMON-LISP-USER package
   0 type ---------> Bit field: #x07
   1 flags --------> Bit field: #x00
   2 hash ---------> Bit field: #x61d7
   3 value --------> ..unbound..
   4 package ------> The COMMON-LISP-USER package
   5 function -----> #<Function BAR>
   6 name ---------> A simple-string (3) "BAR"
   7 plist --------> <...>, a proper list with 2 elements
[1i] CL-USER(19): (symbol-function 'bar)
#<Function BAR>
[1i] CL-USER(20): (symbol-function 'my-new-foo)
Error: Symbol MY-NEW-FOO does not have a function definition.
  [condition type: SIMPLE-ERROR]

Restart actions (select using :continue):
 0: Return to Debug Level 1 (an "abort" restart).
 1: Return to Top Level (an "abort" restart).
 2: Abort entirely from this process.
[2] CL-USER(21): :cont 0
[1i] CL-USER(22): :i q ; get out of the inspection of 'bar
BAR
CL-USER(23): (defun my-new-foo (x) x) ; == cl:identity function

MY-NEW-FOO
CL-USER(24): (symbol-function 'my-new-foo)
#<Interpreted Function MY-NEW-FOO>
CL-USER(25): (compile 'my-new-foo)
MY-NEW-FOO
NIL
NIL
CL-USER(26): (symbol-function 'my-new-foo)
#<Function MY-NEW-FOO>
CL-USER(27): (disassemble 'my-new-foo)
;; disassembly of #<Function MY-NEW-FOO>
;; formals: X

;; code start: #x714242c4:
   0: 83 f9 01    cmpl	ecx,$1
   3: 74 02       jz	7
   5: cd 61       int	$97             ; EXCL::TRAP-ARGERR
   7: 80 7f 97 00 cmpb	[edi-105],$0    ; SYS::C_INTERRUPT
  11: 74 02       jz	15
  13: cd 64       int	$100            ; EXCL::TRAP-SIGNAL-HIT
  15: f8          clc
  16: 8b 75 fc    movl	esi,[ebp-4]
  19: c3          ret
CL-USER(28): (my-new-foo (list 1 2 3))
(1 2 3)
CL-USER(29): 
CL-USER(33): (inspect 'my-new-foo)
The symbol MY-NEW-FOO @ #x714221d7
  which is an INTERNAL symbol in the COMMON-LISP-USER package
   0 type ---------> Bit field: #x07
   1 flags --------> Bit field: #x00
   2 hash ---------> Bit field: #xb1bc
   3 value --------> ..unbound..
   4 package ------> The COMMON-LISP-USER package
   5 function -----> #<Function MY-NEW-FOO>
   6 name ---------> A simple-string (10) "MY-NEW-FOO"
   7 plist --------> (EXCL::.ARGS. NIL), a proper list with 2 elements
[1i] CL-USER(34): :i 5 ; let's look into the function slot
#<Function MY-NEW-FOO>
  lambda-list: (X)
   0 excl-type ----> Bit field: #x08
   1 flags --------> Bit field: #xa8
   2 start --------> Bit field: #x714242c4
   3 hash ---------> Bit field: #x00005b4c
   4 symdef -------> The symbol MY-NEW-FOO
   5 code ---------> simple CODE vector (14) = #(63875 29697 52482...)
   6 formals ------> (X), a proper list with 1 element
   7 cframe-size --> fixnum 0 [#x00000000]
   8 immed-args ---> fixnum 0 [#x00000000]
   9 locals -------> fixnum 0 [#x00000000]
[1i] CL-USER(35): :i 5 ; let's look at the code vector
A simple CODE vector (14) @ #x714242d2
   0-> The field #xf983
   1-> The field #x7401
   2-> The field #xcd02
   3-> The field #x8061
   4-> The field #x977f
   5-> The field #x7400
   6-> The field #xcd02
   7-> The field #xf864
   8-> The field #x758b
   9-> The field #xc3fc
  10-> The field #x0000
  11-> The field #x0000
  12-> The field #x0000
  13-> The field #x0000
[1i] CL-USER(36): :i q ; get out of 'my-new-foo inspection
MY-NEW-FOO
CL-USER(38): 


> That was the OP's question; how to get dynamically generated code
> linked in to a running program. Just saying "use CL" doesn't explain
> the mechanism.

In CL you don't need an explaination of the mechanism unless you are
_implementing_ a Common Lisp.  For the user, you don't need to do
anything other than (compile-file ...) (load ...) to get all the
resources in native compiled form "linked" in and ready to use.  Or
for interactive work, just (compile 'your-foo-here).  That's it.  A
pleasent thing indeed.


/Jon