* load and use a ".o" file? @ 2003-12-18 23:31 lifetime n00b 2003-12-18 23:59 ` Stephen Leake 2003-12-19 1:51 ` tmoran 0 siblings, 2 replies; 29+ messages in thread From: lifetime n00b @ 2003-12-18 23:31 UTC (permalink / raw) I've been trying to educate myself on how to do this, but I can't seem to find the answers I need. In concept, what I need to be able to do seems like it should be simple, and so I'll give a simple example. There is a main program loop already running, and there is a ".o" compiled object file (compiled with gnat) which *hasn't* been linked into the main program, but it has a procedure or function in it I want to call. Let's say it is very simple function which was compiled from this: function HiThere return String is begin return "Hi there!"; end HiThere; Instead of linking this into an executable, it is just compiled with say, "gnatmake -c hithere", to get the "hithere.o" file we're interested in. Now, how can I load this "hithere.o" file into an already running program (without linking it in beforehand) and then call the "HiThere" function from that program (which was also written in Ada)? Would the information in the corresponding .ali file be of use? Maybe I'm thinking about it wrong, and the source which compiles to the object file needs to have some access types or some other mechanisms in place? Note that I'm not talking about using a .dll or .so file, just the .o file. Any guidance would be appreciated. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-18 23:31 load and use a ".o" file? lifetime n00b @ 2003-12-18 23:59 ` Stephen Leake 2003-12-19 1:51 ` tmoran 1 sibling, 0 replies; 29+ messages in thread From: Stephen Leake @ 2003-12-18 23:59 UTC (permalink / raw) To: lifetime n00b; +Cc: comp.lang.ada lifetime n00b <anonymous@yahoo.com> writes: > There is a main program loop already running, and there is a ".o" > compiled object file (compiled with gnat) which *hasn't* been linked > into the main program, but it has a procedure or function in it I want > to call. You need to compile and link it into a "dynamically linked library" (dll on windows, so (shared object) on Unix). Then you need to import your OS's dynamic linking API (which may be done somewhere already), and you're all set. To get more specific, you need to say what OS you are on. And someone else will have to reply; I've not actually done this as yet. -- -- Stephe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-18 23:31 load and use a ".o" file? lifetime n00b 2003-12-18 23:59 ` Stephen Leake @ 2003-12-19 1:51 ` tmoran 2003-12-19 15:28 ` lifetime n00b 1 sibling, 1 reply; 29+ messages in thread From: tmoran @ 2003-12-19 1:51 UTC (permalink / raw) >Let's say it is very simple function which was compiled from this: > >function HiThere return String is >begin > return "Hi there!"; >end HiThere; > >Instead of linking this into an executable, it is just compiled with >say, "gnatmake -c hithere", to get the "hithere.o" file we're interested in. > >Now, how can I load this "hithere.o" file into an already running >program (without linking it in beforehand) and then call the "HiThere" >... >I'm not talking about using a .dll or .so file, just the .o file. If you are not talking about using an existing dynamic linker/loader, then you are clearly talking about developing your own. In your example, HiThere may need to make calls (for memory allocation for the string, for instance) on the Ada Run Time System of the main program. So you will need to load the code from the .o file and then modify call addresses, ie, link, the loaded HiThere function. You should be able to find the information you need by looking at the Gnat source code. What problem are you trying to solve? Perhaps there's an easier way. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-19 1:51 ` tmoran @ 2003-12-19 15:28 ` lifetime n00b 2003-12-19 18:08 ` Stephen Leake 2003-12-19 21:28 ` Simon Wright 0 siblings, 2 replies; 29+ messages in thread From: lifetime n00b @ 2003-12-19 15:28 UTC (permalink / raw) tmoran@acm.org wrote: > If you are not talking about using an existing dynamic linker/loader, > then you are clearly talking about developing your own. In your example, > HiThere may need to make calls (for memory allocation for the string, for > instance) on the Ada Run Time System of the main program. So you will > need to load the code from the .o file and then modify call addresses, ie, > link, the loaded HiThere function. You should be able to find the > information you need by looking at the Gnat source code. > What problem are you trying to solve? Perhaps there's an easier way. Well, the "Big Picture" is that I'm wanting to write a Common Lisp system in Ada (yes I know, it probably sounds like heresy to both the Lisp and Ada camps), and I want to use gnat as the back-end compiler. Contrary to popular opinion, Lisp is not simply an interpreted language. One feature of a complete Common Lisp system is that source files can be compiled and loaded while the top-level system is running. Some implementations (such as CLISP) use byte-code compilation, but most compile to native code. Some use their own built-in compilers, while others first translate the Lisp code to C and use gcc as the back-end compiler and then load and use that compiled file. From what I can tell, it seems that GCL and ECL use gcc to compile the files as dynamic load (.so) files and then call dlopen() on that file. I was thinking that it would be more efficient (and more portable) to not use .dll or .so files, but to load and use the .o object files directly in some way. And if I could figure out how to do it with the example I described in the OP, then I could just expand on that to accomplish what I'm ultimately aiming towards. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-19 15:28 ` lifetime n00b @ 2003-12-19 18:08 ` Stephen Leake 2003-12-20 20:12 ` lifetime n00b 2003-12-19 21:28 ` Simon Wright 1 sibling, 1 reply; 29+ messages in thread From: Stephen Leake @ 2003-12-19 18:08 UTC (permalink / raw) To: comp.lang.ada lifetime n00b <anonymous@yahoo.com> writes: > I was thinking that it would be more efficient (and more portable) to > not use .dll or .so files, but to load and use the .o object files > directly in some way. .dll and .so files are designed to do _precisely_ what you are trying to do; they are the most efficient solution! Hmm. Well, they were designed in the context of a lot of heritage. If you want to design an object file format and dynamic linker from scratch, you might be able to do better. But there's no point; .dll and .so are the only reasonable choices at this point. -- -- Stephe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-19 18:08 ` Stephen Leake @ 2003-12-20 20:12 ` lifetime n00b 2003-12-20 21:15 ` tmoran ` (3 more replies) 0 siblings, 4 replies; 29+ messages in thread From: lifetime n00b @ 2003-12-20 20:12 UTC (permalink / raw) Stephen Leake wrote: > .dll and .so files are designed to do _precisely_ what you are trying > to do; they are the most efficient solution! Well, not _precisely_, since a typical program using the .dll or .so files needs to know about those files when it is compiled itself, right? Maybe I wasn't clear on exactly what I need to do. The program that needs to load and use the object file is *already running* before the object file is compiled. This "main program" has no prior information about the data or routines which may be in the object file it is being told to load. The "main program" operates much like a command prompt, and when (compile-file "hithere") is typed at the prompt, gnat will be called to compile an object file. Then when (load "hithere") is typed at the prompt, the object file is loaded and the data and/or functions in "hithere" are now available to the main program. Some additional information may have to be included in a supporting file produced at the same time the object file is produced, and this supporting file would be loaded when the object file is loaded. That's the idea anyway. But I'm having a difficult time trying to figure out how to accomplish this seemingly straightforward task. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-20 20:12 ` lifetime n00b @ 2003-12-20 21:15 ` tmoran 2003-12-20 23:41 ` lifetime n00b 2003-12-21 11:46 ` Simon Wright ` (2 subsequent siblings) 3 siblings, 1 reply; 29+ messages in thread From: tmoran @ 2003-12-20 21:15 UTC (permalink / raw) >The "main program" operates much like a command prompt, and when >(compile-file "hithere") is typed at the prompt, gnat will be called to >compile an object file. Then when (load "hithere") is typed at the >prompt, the object file is loaded and the data and/or functions in >"hithere" are now available to the main program. Some additional How about making the freshly generated "hithere" file a server program with remotely accessible procedures etc and use Annex E. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-20 21:15 ` tmoran @ 2003-12-20 23:41 ` lifetime n00b 2003-12-21 7:15 ` tmoran 0 siblings, 1 reply; 29+ messages in thread From: lifetime n00b @ 2003-12-20 23:41 UTC (permalink / raw) tmoran@acm.org wrote: > How about making the freshly generated "hithere" file a server program > with remotely accessible procedures etc and use Annex E. That would be horribly inefficient. Conceptually, the contents of these object files become part of the "main program" when they are loaded and share the same address space with the main program. For example, a call to a function normally involves putting the arguments on the stack, executing the function, and reading the return value from a register. In the distributed case, the partition containing the function must be located and placed on the stream, the arguments to the function are placed on the stream, the stream is sent to the receiving partition which executes the function locally, and the result of the function call is placed back on the stream which is then read as the return value. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-20 23:41 ` lifetime n00b @ 2003-12-21 7:15 ` tmoran 0 siblings, 0 replies; 29+ messages in thread From: tmoran @ 2003-12-21 7:15 UTC (permalink / raw) > > with remotely accessible procedures etc and use Annex E. > > That would be horribly inefficient. The question is, would it take too long. If you are going to make billions of these calls and subroutine call efficiency is a real problem, perhaps you should write out an entire new program, including the freshly generated code, and compile that with full global optimization, subroutine inlining, etc. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-20 20:12 ` lifetime n00b 2003-12-20 21:15 ` tmoran @ 2003-12-21 11:46 ` Simon Wright 2003-12-21 13:57 ` Stephen Leake 2003-12-22 15:50 ` Mark H Johnson 3 siblings, 0 replies; 29+ messages in thread From: Simon Wright @ 2003-12-21 11:46 UTC (permalink / raw) lifetime n00b <anonymous@yahoo.com> writes: > Well, not _precisely_, since a typical program using the .dll or .so > files needs to know about those files when it is compiled itself, > right? Maybe I wasn't clear on exactly what I need to do. The > program that needs to load and use the object file is *already > running* before the object file is compiled. This "main program" has > no prior information about the data or routines which may be in the > object file it is being told to load. man dlopen (assuming you're on a Unix box of course). What the equivalent for win32 is I don't know, but I'm sure there is one. -- Simon Wright 100% Ada, no bugs. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-20 20:12 ` lifetime n00b 2003-12-20 21:15 ` tmoran 2003-12-21 11:46 ` Simon Wright @ 2003-12-21 13:57 ` Stephen Leake 2003-12-22 19:29 ` lifetime n00b 2003-12-22 20:49 ` Jon S. Anthony 2003-12-22 15:50 ` Mark H Johnson 3 siblings, 2 replies; 29+ messages in thread From: Stephen Leake @ 2003-12-21 13:57 UTC (permalink / raw) To: lifetime n00b; +Cc: comp.lang.ada lifetime n00b <anonymous@yahoo.com> writes: > Stephen Leake wrote: > > .dll and .so files are designed to do _precisely_ what you are trying > > to do; they are the most efficient solution! > > Well, not _precisely_, since a typical program using the .dll or .so > files needs to know about those files when it is compiled itself, > right? That's one way to use dynamically linked libraries. But if you look under the hood, you will see that is just a variant of what you are doing. When a program that uses dlls starts up, it dynamically links the dll. What you want is a program that dynamically links a dll at some later time. In either case, the dynamic linking is the key. > Maybe I wasn't clear on exactly what I need to do. The program that > needs to load and use the object file is *already running* before > the object file is compiled. This "main program" has no prior > information about the data or routines which may be in the object > file it is being told to load. Well, it can't have absolutely _no_ information. It has to know the list of subroutines (and their parameter profiles) that are in the dll. Or, in COM style, it has to have a way of finding out what subroutines are in the dll; which means it has to know the names of the introspection subroutines in the dll. When you dynamically link the compiled file into the main program, you look up the addresses of those known routines. > The "main program" operates much like a command prompt, and when > (compile-file "hithere") is typed at the prompt, gnat will be called > to compile an object file. Then when (load "hithere") is typed at > the prompt, the object file is loaded and the data and/or functions > in "hithere" are now available to the main program. Yes, that is what I thought you were doing. But how does the main program know the names of the functions in "hithere"? > Some additional information may have to be included in a supporting > file produced at the same time the object file is produced, and this > supporting file would be loaded when the object file is loaded. Ah; perhaps that's the names of the functions? In Windows, that's a .def file, which lists the names of the functions in a dll. > That's the idea anyway. But I'm having a difficult time trying to > figure out how to accomplish this seemingly straightforward task. Go read the manuals on dlls, and dynamic linking in general, and on COM. You are re-inventing the wheel! It may seem straightforward, but it gets quite complicated. Fortunately, it has been solved. Most good ideas have been thought of before :). -- -- Stephe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-21 13:57 ` Stephen Leake @ 2003-12-22 19:29 ` lifetime n00b 2003-12-22 20:49 ` Jon S. Anthony 1 sibling, 0 replies; 29+ messages in thread From: lifetime n00b @ 2003-12-22 19:29 UTC (permalink / raw) Stephen Leake wrote: > You are re-inventing the wheel! No, I'm just trying to find the wheel I need! Honest. If I thought I'd have to reinvent the thing, I'd be the first to point out that I'm the last one qualified to do so, and then I'd just try to find a square I could use instead (in this case, maybe a byte-code VM?). :-) > It may seem straightforward, but it gets quite complicated. So I have found. > Fortunately, it has been solved. I think I may be able to use dlopen and dlsym on Linux with an Ada wrapper, but I was hoping for an Ada-only solution that would be cross-platform (letting gnat take all the nasty details from me). But that's just me hoping for a wheel that can fly... > Most good ideas have been thought of before :). And almost all of them were thought of by Kepler (just don't tell that to Newton)! :) ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-21 13:57 ` Stephen Leake 2003-12-22 19:29 ` lifetime n00b @ 2003-12-22 20:49 ` Jon S. Anthony 2003-12-22 23:15 ` Stephen Leake 1 sibling, 1 reply; 29+ messages in thread From: Jon S. Anthony @ 2003-12-22 20:49 UTC (permalink / raw) Stephen Leake <stephen_leake@acm.org> writes: > lifetime n00b <anonymous@yahoo.com> writes: > ... > > Maybe I wasn't clear on exactly what I need to do. The program that > > needs to load and use the object file is *already running* before > > the object file is compiled. This "main program" has no prior > > information about the data or routines which may be in the object > > file it is being told to load. This sort of thing is perfectly reasonable and something that is extremely useful in a wide range of applications. What I don't understand is why people don't just use technology that has solved this problem since "for ever". No, dll's and so's are not the answer - they are intended to solve a different (related) problem and are at best a poor mans hack for what you are talking about. What you really "need"[1] here is a true dynamic language. > Most good ideas have been thought of before :). Indeed. I see that Greenspun's 10th is alive and well on c.l.a :-|... /Jon ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-22 20:49 ` Jon S. Anthony @ 2003-12-22 23:15 ` Stephen Leake 2003-12-23 1:36 ` tmoran 2003-12-27 22:55 ` Jon S. Anthony 0 siblings, 2 replies; 29+ messages in thread From: Stephen Leake @ 2003-12-22 23:15 UTC (permalink / raw) To: comp.lang.ada j-anthony@rcn.com (Jon S. Anthony) writes: > This sort of thing is perfectly reasonable and something that is > extremely useful in a wide range of applications. > > What I don't understand is why people don't just use technology that > has solved this problem since "for ever". No, dll's and so's are not > the answer - they are intended to solve a different (related) problem > and are at best a poor mans hack for what you are talking about. > > What you really "need"[1] here is a true dynamic language. Ok, I'll bite. What's a "true dynamic language"? Do you mean "interpreted from source", like some versions of Basic, and most shell languages? Or "interpreted from some intermediate byte code", like most Visual Basic, Lisp implementations, Java, C#, etc? Or something else? All interpreters make the job of adding random code at run-time much easier. -- -- Stephe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-22 23:15 ` Stephen Leake @ 2003-12-23 1:36 ` tmoran 2003-12-27 22:55 ` Jon S. Anthony 1 sibling, 0 replies; 29+ messages in thread From: tmoran @ 2003-12-23 1:36 UTC (permalink / raw) >All interpreters make the job of adding random code at run-time much >easier. If I understand correctly, the problem is not in adding code, but in finding code. The compiler has put the code, in some form, in a .o file and the OP wants to pluck it out so he can call it - perhaps with an indirect jump, ie, a software simulation of Multics et al. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-22 23:15 ` Stephen Leake 2003-12-23 1:36 ` tmoran @ 2003-12-27 22:55 ` Jon S. Anthony 2003-12-28 3:28 ` Stephen Leake 1 sibling, 1 reply; 29+ messages in thread From: Jon S. Anthony @ 2003-12-27 22:55 UTC (permalink / raw) Stephen Leake <stephen_leake@acm.org> writes: > j-anthony@rcn.com (Jon S. Anthony) writes: > > > This sort of thing is perfectly reasonable and something that is > > extremely useful in a wide range of applications. > > > > What I don't understand is why people don't just use technology that > > has solved this problem since "for ever". No, dll's and so's are not > > the answer - they are intended to solve a different (related) problem > > and are at best a poor mans hack for what you are talking about. > > > > What you really "need"[1] here is a true dynamic language. > > Ok, I'll bite. What's a "true dynamic language"? It is true that many folks will/do have different ideas about what "true dynamic language" means. In this respect I fear it is a bit like "true object oriented language". One characteristic that is neither necessary nor sufficient for any notion of this is that it be an "interpreted language". While some languages (for example, most scripting languages) do imply (mostly because the language specification _is_ the implementation) interpretation as part of their definition, "interpreted" interpreted properly is really an _implementation_ issue, _not_ a language issue. I think most people in the "dynamic community" would say you need at least the following characteristics to be "truly dynamic": 1. Dynamic typing. This is, strong typing on _objects_ as differentiated from "variables/locations". 2. Complete introspection of the language's computation and type model. This is directly and explicitly available at the level of the programmer (not just the implementer). NOTE: This does _not_ mean or imply access to the underlying _implementation_. Some things this _does_ imply: * New function objects may be added to a running system at any time and existing functions may be so modified at any time. * The ability to create new types and/or classes at runtime; modify existing classes (with complete instance migration). A couple examples which fully have this would be Common Lisp and Smalltalk. Neither are "interpreter" based[1]. Most Common Lisp implementations have optimizing native compilers that typically produce code within about 1.5-2.5 of well crafted C code[2]. So, "interpretation" of either source or byte code is not a relevant factor/issue. For more information, here are a couple of links: "Open Source/Free Software" Implementations: http://www.cliki.net/Common%20Lisp%20implementation Commercial Implementations: http://alu.cliki.net/Implementation General community information: http://www.cliki.net/index and, of course, c.l.l Here are a couple very simple examples concerning the issue at hand of the OP. These are done in Allegro Common Lisp at the REPL (read eval print loop), typically this sort of interaction would be through the IDE. CL-USER(1): (defun foo (l r) (declare (fixnum l r) (optimize (speed 3 safety 1 debug 1))) (+ l r)) => FOO CL-USER(2): (compile 'foo) => FOO CL-USER(3): (disassemble 'foo) ; Just to show we are native (on Intel...) ;; disassembly of #<Function FOO> ;; formals: L R ;; code start: #x7140a65c: 0: 83 f9 02 cmpl ecx,$2 3: 74 02 jz 7 5: cd 61 int $97 ; EXCL::TRAP-ARGERR 7: 80 7f 97 00 cmpb [edi-105],$0 ; SYS::C_INTERRUPT 11: 74 02 jz 15 13: cd 64 int $100 ; EXCL::TRAP-SIGNAL-HIT 15: 03 c2 addl eax,edx 17: f8 clc 18: 8b 75 fc movl esi,[ebp-4] 21: c3 ret This example shows compiling and loading a set of resources from a file under program control. ;;;--------------------junk.cl------------ (in-package :cl-user) (defun foo (l r) (declare (fixnum l r) (optimize (speed 3 safety 1 debug 1))) (+ l r)) (defun bar (msg) (format nil "Here's a message: ~A" msg)) ;;;-----------------end junk.cl------------ ;;;---------------------load-from.cl----------- (in-package :cl-user) ;;; Load the function definitions (and other forms) in the file ;;; identified by FILESPEC (a pathname or a string which is a legal ;;; filespec for the underlying OS filesystem). Compile the file ;;; before loading it. ;;; ;;; WHEN-UNDEFINED indicates a level of conditionality for compiling ;;; and loading the file. If not given, or explicitly set to NIL, ;;; unconditionally compile and load the file. ;;; ;;; If given, it must be a list of function names, which are tested to ;;; see if they are currently defined. If _none_ of them are, then ;;; compile and load the file. ;;; ;;; NOTE: no current provision for checking to see if the definitions ;;; in the file are of a later time stamp than the current existing ;;; function definitions (which could be used to conditionally compile ;;; and load when definitions are superceded). ;;; ;;; Errors and serious conditions are handled by sending a message on ;;; the condition to the standard error stream. ;;; ;;; Returns t if file is successfully compiled and loaded, :noaction ;;; if no action is taken (see above), :error if an error is signaled ;;; or :serious if a serious condition is signaled. ;;; (defun load-from (filespec &key (when-undefined nil)) (assert (listp when-undefined)) (handler-case (if (or (eql when-undefined nil) (reduce #'(lambda (x y) (and x (not (fboundp y)))) when-undefined :initial-value t)) (load (compile-file filespec)) :noaction) ;; Exceptions ;; (error (condition) (format *error-output* "LOAD-FROM crashed with ~S" condition) :error) (serious-condition (condition) (format *error-output* "LOAD-FROM questionable: ~S" condition) :serious))) ;;;-----------------end load-from.cl----------- ;;; Load the previously compiled load-from object CL-USER(1): (load "load-from") ; Fast loading load-from.fasl T CL-USER(2): (foo 1 2) Error: attempt to call `FOO' which is an undefined function. [condition type: UNDEFINED-FUNCTION] CL-USER(4): (fboundp 'foo) NIL foo is undefined... CL-USER(5): (load-from "./junk.cl" :when-undefined '(load-from)) :NOACTION Since load-from is already defined, this does nothing per the spec. CL-USER(6): (load-from "./junk.cl" :when-undefined '(foo bar)) ;;; Compiling file ./junk.cl ;;; Writing fasl file ./junk.fasl ;;; Fasl write complete ; Fast loading /opt/GoldenThread/ExampleApps/ECDemo/junk.fasl T CL-USER(7): (foo 1 2) 3 CL-USER(8): (bar "Hello world!") "Here's a message: Hello world!" > Do you mean "interpreted from source" No, as this is an implementation issue. > Or "interpreted from some intermediate byte code", like most Visual > Basic, Lisp implementations, Java, C#, etc? No, as this is an implementation issue. BTW, _most_ Lisp implementations have _native compilers_. Some (e.g., CMUCL) _only_ have native compilers - no interpreter at all. > Or something else? Yes, as described above. > All interpreters make the job of adding random code at run-time much > easier. Actually, this only seems to be true; with fully compiled versions _this aspect_ is just as easy. The hard part, of course, is still the bits that make up the difference between a compiler and interpreter. /Jon [1] The ANSI Common Lisp specification http://www.lispworks.com/reference/HyperSpec/Front/index.htm requires a compiler, though not necessarily a native code generator. I'm much more familiar with Common Lisp than Smalltalk, but VisualWorks, Squeak, and VisualAge all have JIT compilers. Python offers some of this as well, but the "definition/reference implementations" are interpreted. [2] There are several types of basic block structures where CL can perform better than such C code. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-27 22:55 ` Jon S. Anthony @ 2003-12-28 3:28 ` Stephen Leake 2003-12-28 16:14 ` Georg Bauhaus 2003-12-29 22:42 ` Jon S. Anthony 0 siblings, 2 replies; 29+ messages in thread From: Stephen Leake @ 2003-12-28 3:28 UTC (permalink / raw) To: comp.lang.ada j-anthony@rcn.com (Jon S. Anthony) writes: > Stephen Leake <stephen_leake@acm.org> writes: > > > j-anthony@rcn.com (Jon S. Anthony) writes: > > > > > This sort of thing is perfectly reasonable and something that is > > > extremely useful in a wide range of applications. > > > > > > What I don't understand is why people don't just use technology that > > > has solved this problem since "for ever". No, dll's and so's are not > > > the answer - they are intended to solve a different (related) problem > > > and are at best a poor mans hack for what you are talking about. > > > > > > What you really "need"[1] here is a true dynamic language. > > > > Ok, I'll bite. What's a "true dynamic language"? > > It is true that many folks will/do have different ideas about what > "true dynamic language" means. In this respect I fear it is a bit > like "true object oriented language". That's why I asked :). > I think most people in the "dynamic community" would say you need at > least the following characteristics to be "truly dynamic": > > 1. Dynamic typing. This is, strong typing on _objects_ as > differentiated from "variables/locations". Ok. I don't think that impacts on the issue of adding new code at runtime. > 2. Complete introspection of the language's computation and type > model. This is directly and explicitly available at the level of the > programmer (not just the implementer). NOTE: This does _not_ mean or > imply access to the underlying _implementation_. > > Some things this _does_ imply: > > * New function objects may be added to a running system at any time > and existing functions may be so modified at any time. Hmm. The English meaning of "introspection" is "to examine oneself"; that's not the same as creating new code. But you get to use the term the way you want to ... > * The ability to create new types and/or classes at runtime; > modify existing classes (with complete instance migration). > > > A couple examples which fully have this would be Common Lisp and > Smalltalk. Neither are "interpreter" based[1]. Most Common Lisp > implementations have optimizing native compilers that typically > produce code within about 1.5-2.5 of well crafted C code[2]. Hmm. Do those compiled versions include the ability to create new code at runtime, where the new code is also compiled, not interpreted? I'd be very surprised if that were true (I have not used a system where that is true, but I have used only a couple Lisps, and no Smalltalks). If it is true, what is the linking mechanism used? > <snip> Here are a couple very simple examples concerning the issue > at hand of the OP. These are done in Allegro Common Lisp at the REPL > (read eval print loop), typically this sort of interaction would be > through the IDE. > > CL-USER(1): > (defun foo (l r) > (declare (fixnum l r) (optimize (speed 3 safety 1 debug 1))) > (+ l r)) > => FOO > > CL-USER(2): (compile 'foo) > => FOO > > CL-USER(3): (disassemble 'foo) ; Just to show we are native (on Intel...) > ;; disassembly of #<Function FOO> > ;; formals: L R > > ;; code start: #x7140a65c: > 0: 83 f9 02 cmpl ecx,$2 > 3: 74 02 jz 7 > 5: cd 61 int $97 ; EXCL::TRAP-ARGERR > 7: 80 7f 97 00 cmpb [edi-105],$0 ; SYS::C_INTERRUPT > 11: 74 02 jz 15 > 13: cd 64 int $100 ; EXCL::TRAP-SIGNAL-HIT > 15: 03 c2 addl eax,edx > 17: f8 clc > 18: 8b 75 fc movl esi,[ebp-4] > 21: c3 ret Hmm. You don't actually demonstrate calling foo, but I'll grant you that :). Ok, I stand corrected. Interpretation is not necessary for dynamically generated and linked code. But I'd still like to understand what the linking process implied by (compile 'foo). As far as I can see, it must do the equivalent of Windows DLL or Unix so. Just because that mechanism is hidden from the user (as it should be) doesn't mean it isn't there! That was the OP's question; how to get dynamically generated code linked in to a running program. Just saying "use CL" doesn't explain the mechanism. -- -- Stephe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-28 3:28 ` Stephen Leake @ 2003-12-28 16:14 ` Georg Bauhaus 2003-12-29 22:45 ` Jon S. Anthony 2003-12-29 22:42 ` Jon S. Anthony 1 sibling, 1 reply; 29+ messages in thread From: Georg Bauhaus @ 2003-12-28 16:14 UTC (permalink / raw) Stephen Leake <stephen_leake@acm.org> wrote: : Hmm. Do those compiled versions include the ability to create new code : at runtime, where the new code is also compiled, not interpreted? If I have understood Jon correctly, there are Lisp compilers without an interpreter, and with the ability to load compiled code, so I'd guess, yes. (I think I've seen the thing, too.) For SNOBOL4, a minimal RE(P)L which runs the compiler at run time is read :<code(input ' :(read)')> end SPITBOL is a "true" SNOBOL4 compiler afaik, so it might offer the same capability[*]. (If you are interested, the :<...> triggers a jump to the beginning of ..., which is compiled code() from ``input ' :(read)'''; input reads what you type, ' :(read)' is an unconditional jump to label 'read', appended to every such input.) [*]From the SPITBOL description at www.snobol4.com: "New program fragments can be compiled during program execution... For very complex problems, your program can "evovle" and modify itself during execution in response to the data it is processing. New patterns can be built during execution." ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-28 16:14 ` Georg Bauhaus @ 2003-12-29 22:45 ` Jon S. Anthony 0 siblings, 0 replies; 29+ messages in thread From: Jon S. Anthony @ 2003-12-29 22:45 UTC (permalink / raw) Georg Bauhaus <sb463ba@l1-hrz.uni-duisburg.de> writes: > Stephen Leake <stephen_leake@acm.org> wrote: > : Hmm. Do those compiled versions include the ability to create new code > : at runtime, where the new code is also compiled, not interpreted? > > If I have understood Jon correctly, there are Lisp compilers without > an interpreter, and with the ability to load compiled code, so > I'd guess, yes. Yes. Though also having an interpreter as part of the implementation does not change this (it's orthogonal to the discussion). > (I think I've seen the thing, too.) Most do this, so you probably did :-) /Jon ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-28 3:28 ` Stephen Leake 2003-12-28 16:14 ` Georg Bauhaus @ 2003-12-29 22:42 ` Jon S. Anthony 2003-12-30 15:17 ` lifetime n00b 2003-12-30 16:56 ` Stephen Leake 1 sibling, 2 replies; 29+ messages in thread From: Jon S. Anthony @ 2003-12-29 22:42 UTC (permalink / raw) Stephen Leake <stephen_leake@acm.org> writes: > (Jon S. Anthony) writes: > > > Stephen Leake <stephen_leake@acm.org> writes: > > > > > (Jon S. Anthony) writes: ... > > I think most people in the "dynamic community" would say you need at > > least the following characteristics to be "truly dynamic": > > > > 1. Dynamic typing. This is, strong typing on _objects_ as > > differentiated from "variables/locations". > > Ok. I don't think that impacts on the issue of adding new code at runtime. Actually it does, but in non obvious ways. For this discussion, it is safe to ignore it. > > 2. Complete introspection of the language's computation and type > > model. This is directly and explicitly available at the level of the > > programmer (not just the implementer). NOTE: This does _not_ mean or > > imply access to the underlying _implementation_. > > > > Some things this _does_ imply: > > > > * New function objects may be added to a running system at any time > > and existing functions may be so modified at any time. > > Hmm. The English meaning of "introspection" is "to examine oneself"; > that's not the same as creating new code. But you get to use the term > the way you want to ... Fair enough; while the explicit addition of "self modifying" would be clearer for c.l.a, I believe that is taken for granted in the "dynamic community". It is not always easy to clearly change gears in usenet discussions... > > * The ability to create new types and/or classes at runtime; > > modify existing classes (with complete instance migration). > > > > > > A couple examples which fully have this would be Common Lisp and > > Smalltalk. Neither are "interpreter" based[1]. Most Common Lisp > > implementations have optimizing native compilers that typically > > produce code within about 1.5-2.5 of well crafted C code[2]. > > Hmm. Do those compiled versions include the ability to create new code > at runtime, where the new code is also compiled, not interpreted? Yes. > I'd be very surprised if that were true (I have not used a system > where that is true, but I have used only a couple Lisps, and no > Smalltalks). If it is true, what is the linking mechanism used? It is a pleasent surprise, I hope. The "linking" mechanism is simply resolution on a symbol's function slot. This is part of the introspective aspect of a Lisp system. The native code is "just" placed in a code vector. > > CL-USER(1): > > (defun foo (l r) > > (declare (fixnum l r) (optimize (speed 3 safety 1 debug 1))) > > (+ l r)) > > => FOO > > > > CL-USER(2): (compile 'foo) > > => FOO > > > > CL-USER(3): (disassemble 'foo) ; Just to show we are native (on Intel...) > > ;; disassembly of #<Function FOO> > > ;; formals: L R > > > > ;; code start: #x7140a65c: > > 0: 83 f9 02 cmpl ecx,$2 > > 3: 74 02 jz 7 > > 5: cd 61 int $97 ; EXCL::TRAP-ARGERR > > 7: 80 7f 97 00 cmpb [edi-105],$0 ; SYS::C_INTERRUPT > > 11: 74 02 jz 15 > > 13: cd 64 int $100 ; EXCL::TRAP-SIGNAL-HIT > > 15: 03 c2 addl eax,edx > > 17: f8 clc > > 18: 8b 75 fc movl esi,[ebp-4] > > 21: c3 ret > > Hmm. You don't actually demonstrate calling foo, but I'll grant you > that :). That was a little further below in the example which compiles and loads foo from a file. > But I'd still like to understand what the linking process implied by > (compile 'foo). As far as I can see, it must do the equivalent of > Windows DLL or Unix so. No, this is part of the underlying introspective environment. It is actually much simpler than the contortions involved in DLL/so type stuff. It is also fine grained - a function at a time or even a functions symbol at a time. > Just because that mechanism is hidden from the user (as it should > be) doesn't mean it isn't there! Yes, it is there, and you can even get at the important user (programmer) level bits yourself. Here is a few interactions after the compile and load of the junk.cl file: CL-USER(18): (inspect 'bar) The symbol BAR @ #x7140990f which is an INTERNAL symbol in the COMMON-LISP-USER package 0 type ---------> Bit field: #x07 1 flags --------> Bit field: #x00 2 hash ---------> Bit field: #x61d7 3 value --------> ..unbound.. 4 package ------> The COMMON-LISP-USER package 5 function -----> #<Function BAR> 6 name ---------> A simple-string (3) "BAR" 7 plist --------> <...>, a proper list with 2 elements [1i] CL-USER(19): (symbol-function 'bar) #<Function BAR> [1i] CL-USER(20): (symbol-function 'my-new-foo) Error: Symbol MY-NEW-FOO does not have a function definition. [condition type: SIMPLE-ERROR] Restart actions (select using :continue): 0: Return to Debug Level 1 (an "abort" restart). 1: Return to Top Level (an "abort" restart). 2: Abort entirely from this process. [2] CL-USER(21): :cont 0 [1i] CL-USER(22): :i q ; get out of the inspection of 'bar BAR CL-USER(23): (defun my-new-foo (x) x) ; == cl:identity function MY-NEW-FOO CL-USER(24): (symbol-function 'my-new-foo) #<Interpreted Function MY-NEW-FOO> CL-USER(25): (compile 'my-new-foo) MY-NEW-FOO NIL NIL CL-USER(26): (symbol-function 'my-new-foo) #<Function MY-NEW-FOO> CL-USER(27): (disassemble 'my-new-foo) ;; disassembly of #<Function MY-NEW-FOO> ;; formals: X ;; code start: #x714242c4: 0: 83 f9 01 cmpl ecx,$1 3: 74 02 jz 7 5: cd 61 int $97 ; EXCL::TRAP-ARGERR 7: 80 7f 97 00 cmpb [edi-105],$0 ; SYS::C_INTERRUPT 11: 74 02 jz 15 13: cd 64 int $100 ; EXCL::TRAP-SIGNAL-HIT 15: f8 clc 16: 8b 75 fc movl esi,[ebp-4] 19: c3 ret CL-USER(28): (my-new-foo (list 1 2 3)) (1 2 3) CL-USER(29): CL-USER(33): (inspect 'my-new-foo) The symbol MY-NEW-FOO @ #x714221d7 which is an INTERNAL symbol in the COMMON-LISP-USER package 0 type ---------> Bit field: #x07 1 flags --------> Bit field: #x00 2 hash ---------> Bit field: #xb1bc 3 value --------> ..unbound.. 4 package ------> The COMMON-LISP-USER package 5 function -----> #<Function MY-NEW-FOO> 6 name ---------> A simple-string (10) "MY-NEW-FOO" 7 plist --------> (EXCL::.ARGS. NIL), a proper list with 2 elements [1i] CL-USER(34): :i 5 ; let's look into the function slot #<Function MY-NEW-FOO> lambda-list: (X) 0 excl-type ----> Bit field: #x08 1 flags --------> Bit field: #xa8 2 start --------> Bit field: #x714242c4 3 hash ---------> Bit field: #x00005b4c 4 symdef -------> The symbol MY-NEW-FOO 5 code ---------> simple CODE vector (14) = #(63875 29697 52482...) 6 formals ------> (X), a proper list with 1 element 7 cframe-size --> fixnum 0 [#x00000000] 8 immed-args ---> fixnum 0 [#x00000000] 9 locals -------> fixnum 0 [#x00000000] [1i] CL-USER(35): :i 5 ; let's look at the code vector A simple CODE vector (14) @ #x714242d2 0-> The field #xf983 1-> The field #x7401 2-> The field #xcd02 3-> The field #x8061 4-> The field #x977f 5-> The field #x7400 6-> The field #xcd02 7-> The field #xf864 8-> The field #x758b 9-> The field #xc3fc 10-> The field #x0000 11-> The field #x0000 12-> The field #x0000 13-> The field #x0000 [1i] CL-USER(36): :i q ; get out of 'my-new-foo inspection MY-NEW-FOO CL-USER(38): > That was the OP's question; how to get dynamically generated code > linked in to a running program. Just saying "use CL" doesn't explain > the mechanism. In CL you don't need an explaination of the mechanism unless you are _implementing_ a Common Lisp. For the user, you don't need to do anything other than (compile-file ...) (load ...) to get all the resources in native compiled form "linked" in and ready to use. Or for interactive work, just (compile 'your-foo-here). That's it. A pleasent thing indeed. /Jon ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-29 22:42 ` Jon S. Anthony @ 2003-12-30 15:17 ` lifetime n00b 2003-12-30 16:56 ` Stephen Leake 1 sibling, 0 replies; 29+ messages in thread From: lifetime n00b @ 2003-12-30 15:17 UTC (permalink / raw) Jon S. Anthony wrote: > In CL you don't need an explaination of the mechanism unless you are > _implementing_ a Common Lisp. For the user, you don't need to do > anything other than (compile-file ...) (load ...) to get all the > resources in native compiled form "linked" in and ready to use. Or > for interactive work, just (compile 'your-foo-here). That's it. A > pleasent thing indeed. See, I just *knew* this was going to turn out more difficult than it sounded! :-) First let me say I am not trying to build a complete "Common Lisp" implementation, at least not yet. Common Lisp is similar to Ada in that both language definitions are very rich and include many "libraries" as part of the definition itself that other languages have to add on as third party packages. However, one of the nicest features of Lisp is that the fundamental syntax of the language is extremely elegant and simple. Right now, it's just this elegant and simple core that I am trying to implement, using gnat as the machine-code compiler. It's like a Siren, calling me into the deep waters. "Look how beautiful I am! Of course you can obtain me, just come a little closer..." Once I have this core actually working, maybe then I can begin to think in terms of adding the myriad functions and operators that make up the rest of the Common Lisp definition. And I realize that (compile ...) and (load ...) are not part of the core syntax but rather part of the "extras" that make up a complete Common Lisp. However, without these, there is not much point in continuing on, since the whole point of using gnat as the compiler is in the context of these functions. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-29 22:42 ` Jon S. Anthony 2003-12-30 15:17 ` lifetime n00b @ 2003-12-30 16:56 ` Stephen Leake 1 sibling, 0 replies; 29+ messages in thread From: Stephen Leake @ 2003-12-30 16:56 UTC (permalink / raw) To: comp.lang.ada j-anthony@rcn.com (Jon S. Anthony) writes: > Stephen Leake <stephen_leake@acm.org> writes: > > > But I'd still like to understand what the linking process implied by > > (compile 'foo). As far as I can see, it must do the equivalent of > > Windows DLL or Unix so. > > No, this is part of the underlying introspective environment. It is > actually much simpler than the contortions involved in DLL/so type > stuff. It is also fine grained - a function at a time or even a > functions symbol at a time. Well, ok. I don't have a DLL manual handy. Fine-grained would be nice. Maybe sometime I'll find the time to read up on Lisp implementations. I suspect lambda binding, which is very dynamic, actually simplifies the issue of dynamic code loading. > In CL you don't need an explaination of the mechanism unless you are > _implementing_ a Common Lisp. Ok. I'm interested in the implementation details. > For the user, you don't need to do anything other than (compile-file > ...) (load ...) to get all the resources in native compiled form > "linked" in and ready to use. Or for interactive work, just (compile > 'your-foo-here). That's it. A pleasent thing indeed. Right. In VxWorks, you get almost the same convenience, except you have to do the compile on the host, and execute on the target. -- -- Stephe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-20 20:12 ` lifetime n00b ` (2 preceding siblings ...) 2003-12-21 13:57 ` Stephen Leake @ 2003-12-22 15:50 ` Mark H Johnson 2003-12-22 19:46 ` lifetime n00b 3 siblings, 1 reply; 29+ messages in thread From: Mark H Johnson @ 2003-12-22 15:50 UTC (permalink / raw) lifetime n00b wrote: > Stephen Leake wrote: > >> .dll and .so files are designed to do _precisely_ what you are trying >> to do; they are the most efficient solution! > > > Well, not _precisely_, since a typical program using the .dll or .so > files needs to know about those files when it is compiled itself, right? > Maybe I wasn't clear on exactly what I need to do. The program that > needs to load and use the object file is *already running* before the > object file is compiled. This "main program" has no prior information > about the data or routines which may be in the object file it is being > told to load. > What you are describing sounds a lot like the mechanism used on Multics for dynamic linking of object files. There is a better explanation in the references at http://www.multicians.org/ or by reading "The Multics System: An Examination of its Structure" by E.I. Organick but a quick summary follows. When a file is compiled on Multics, an object file is created. Part of the data in that object file is a list of public symbols, including entry points. For a subroutine named xyz, compiled from xyz.fortran, the entry point would simply be xyz. Let's say you have a main program main, calling xyz. This is an external reference to xyz (since it is not found in the current segment), and the first call works something like this: - call xyz (dispatched to an activation function) - the activation function checks first the active segments for a symbol within them called xyz. If found, it fixes up the dispatch to go directly to that xyz and resumes execution. - if not in an active segment, it uses the search path (similar to PATH on Unix) to find a file named xyz. If found, it loads that file into a segment, makes it active, and does the work of the second step. - if no such file is found, it calls the command line interpreter with an error status. At this point, you can fix your search path, create the object file xyz, start the debugger, or abandon execution. If you either of the first two, you can resume and the attempt to load xyz repeats. The subsequent calls to xyz is done just like any other cross segment function call, basically running at (or near) full speed. It is this part that eliminates most of the concerns about performance since the expensive look up is only done once per symbol. This capability, and I/O redirection are two of the things I really miss from Multics. The development flexibility provided were great to enhance the performance of developers and the overhead was quite modest with the hardware provided. The lack of effective paged / segmentation hardware is one thing that makes this hard to implement in software. You can probably get close if you modify the build of objects to generate .so's instead of .o's and then put a wrapper around dlopen to do the semantics you are looking for. If you use GNAT, you could probably even bury this in the runtime by adding a pragma (or command line option) to denote external references as dynamic calls. If you get this to work, I'd like to hear about it. --Mark ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-22 15:50 ` Mark H Johnson @ 2003-12-22 19:46 ` lifetime n00b 2003-12-22 22:58 ` Mark H Johnson 0 siblings, 1 reply; 29+ messages in thread From: lifetime n00b @ 2003-12-22 19:46 UTC (permalink / raw) Mark H Johnson wrote: > - call xyz (dispatched to an activation function) > - the activation function checks first the active segments for a symbol > within them called xyz. If found, it fixes up the dispatch to go > directly to that xyz and resumes execution. > - if not in an active segment, it uses the search path (similar to PATH > on Unix) to find a file named xyz. If found, it loads that file into a > segment, makes it active, and does the work of the second step. > - if no such file is found, it calls the command line interpreter with > an error status. This actually sounds a lot like using a distributed system (Annex E) with both active and passive partitions on the same local machine, though probably more efficient. > The subsequent calls to xyz is done just like any other cross segment > function call, basically running at (or near) full speed. It is this > part that eliminates most of the concerns about performance since the > expensive look up is only done once per symbol. Unfortunately, I won't be programming this for Multics. I may need to suck it up and just do some benchmarks using the Annex E method. Passing data, arguments, and return values along streams between paritions just to call a function sure seems like a huge performance drain, but maybe in practice it's not as bad as I imagine. > If you get this to work, I'd like to hear about it. Me too! :) ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-22 19:46 ` lifetime n00b @ 2003-12-22 22:58 ` Mark H Johnson 2003-12-23 17:48 ` Robert I. Eachus 0 siblings, 1 reply; 29+ messages in thread From: Mark H Johnson @ 2003-12-22 22:58 UTC (permalink / raw) lifetime n00b wrote: > Mark H Johnson wrote: > >> - call xyz (dispatched to an activation function) >> - the activation function checks first the active segments for a >> symbol within them called xyz. If found, it fixes up the dispatch to >> go directly to that xyz and resumes execution. >> - if not in an active segment, it uses the search path (similar to >> PATH on Unix) to find a file named xyz. If found, it loads that file >> into a segment, makes it active, and does the work of the second step. >> - if no such file is found, it calls the command line interpreter >> with an error status. > > > This actually sounds a lot like using a distributed system (Annex E) > with both active and passive partitions on the same local machine, > though probably more efficient. > No, they were actually subroutine calls. The compiler generated slightly different code for an "in segment" call and a "cross segment" call due to the indirection. The glue code (to do the lookup) was basically a system service done by the OS to facilitate activation of new segments. By the way, when I say segment - I mean a segment of the same address space, not a partition. Think to the 386 processor where you have 32 k segments in certain processing modes. On Multics, the segments were 256 Kwords (1 Mbyte) in size which worked quite well in practice. --Mark ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-22 22:58 ` Mark H Johnson @ 2003-12-23 17:48 ` Robert I. Eachus 2003-12-23 17:59 ` Mark H Johnson 0 siblings, 1 reply; 29+ messages in thread From: Robert I. Eachus @ 2003-12-23 17:48 UTC (permalink / raw) Mark H Johnson wrote: > No, they were actually subroutine calls. The compiler generated slightly > different code for an "in segment" call and a "cross segment" call due > to the indirection. The glue code (to do the lookup) was basically a > system service done by the OS to facilitate activation of new segments. Technically not correct. The compiler could "in line" some local calls, but the binder couldn't. What happened was that when a segment called an entry point for which the local link was null, it trapped. The trap routine tried to snap the link using the current environment, and if it failed, bounced out to the user. You literally could, when an error of this type occurred, change the search paths, or write the missing routine, compile it, then continue from the condition. Done it myself.... > By the way, when I say segment - I mean a segment of the same address > space, not a partition. Think to the 386 processor where you have 32 k > segments in certain processing modes. On Multics, the segments were 256 > Kwords (1 Mbyte) in size which worked quite well in practice. Correct. As I said in another post, there were multisegment files which allowed you to have objects larger than 1 Meg. But in practice they were very infrequently used. -- Robert I. Eachus "The war on terror is a different kind of war, waged capture by capture, cell by cell, and victory by victory. Our security is assured by our perseverance and by our sure belief in the success of liberty." -- George W. Bush ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-23 17:48 ` Robert I. Eachus @ 2003-12-23 17:59 ` Mark H Johnson 2003-12-23 21:53 ` Robert I. Eachus 0 siblings, 1 reply; 29+ messages in thread From: Mark H Johnson @ 2003-12-23 17:59 UTC (permalink / raw) Robert I. Eachus wrote: > Mark H Johnson wrote: > >> No, they were actually subroutine calls. The compiler generated >> slightly different code for an "in segment" call and a "cross segment" >> call due to the indirection. The glue code (to do the lookup) was >> basically a system service done by the OS to facilitate activation of >> new segments. > > > Technically not correct. The compiler could "in line" some local calls, > but the binder couldn't. I did not mention the binder (which in many ways I equate to having a linker on Unix), just the compiler. > What happened was that when a segment called > an entry point for which the local link was null, it trapped. The trap > routine tried to snap the link using the current environment, and if it > failed, bounced out to the user. You literally could, when an error of > this type occurred, change the search paths, or write the missing > routine, compile it, then continue from the condition. Done it myself.... > Me to. That is why I described that mechanism in terms the OP would likely understand in a previous message. --Mark ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-23 17:59 ` Mark H Johnson @ 2003-12-23 21:53 ` Robert I. Eachus 0 siblings, 0 replies; 29+ messages in thread From: Robert I. Eachus @ 2003-12-23 21:53 UTC (permalink / raw) Mark H Johnson wrote: > Robert I. Eachus wrote: > >> Technically not correct. The compiler could "in line" some local >> calls, but the binder couldn't. > > I did not mention the binder (which in many ways I equate to having a > linker on Unix), just the compiler. I was not trying to be confrontational or whatever. My point was, that in segments created by the binder, all that was done was to provide a (static, code-relative)target for the call. It didn't change the calling code at all. So if you had a program compiled as a collection of separate segments, and ran it that way, the first time each entry point was used, the link was snapped, and from then on you had no additional run-time overhead. If you used the binder, it eliminated some of the run-time link snapping, but the code was exactly as efficient as if it had been compiled from one huge source file. > Me to. That is why I described that mechanism in terms the OP would > likely understand in a previous message. I still miss Multics. I had accounts on several Multics machines when I worked at Honeywell. When I moved to Stratus, even though Stratus was a 'spritual' descendent of Multics, I kept an account on MIT_Multics. (I had some Multics PL/I tools I wanted to port to VOS.) But it has been fifteen years now, and I still don't have a development environment as good as Multics. However, Ada is now a much better language than PL/I. ;-) -- Robert I. Eachus "The war on terror is a different kind of war, waged capture by capture, cell by cell, and victory by victory. Our security is assured by our perseverance and by our sure belief in the success of liberty." -- George W. Bush ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: load and use a ".o" file? 2003-12-19 15:28 ` lifetime n00b 2003-12-19 18:08 ` Stephen Leake @ 2003-12-19 21:28 ` Simon Wright 1 sibling, 0 replies; 29+ messages in thread From: Simon Wright @ 2003-12-19 21:28 UTC (permalink / raw) lifetime n00b <anonymous@yahoo.com> writes: > I was thinking that it would be more efficient (and more portable) > to not use .dll or .so files, but to load and use the .o object > files directly in some way. And if I could figure out how to do it > with the example I described in the OP, then I could just expand on > that to accomplish what I'm ultimately aiming towards. I realise it is most unlikely that your target OS is VxWorks, but there may be some help to be gleaned .. The way you build Ada for VxWorks (well, with GNAT) is * you do the bind as normal, this generates (among other things) the elaboration code in adainit and a call to it followed by a call to your program entry point * the link is done (like any other VxWorks link) as a partial link (I think that's the right word, the basic command is ld -r), so that no VxWorks runtime is included but the GNAT runtime is * the result is downloaded to the target and relocations resolved there, against the symbol table (I am a little hazy about exact details here, but it is certainly possible to do this download from eg flash memory, so the symbols must be resident). For C under VxWorks there is no elaboration so you can just download a .o. For Ada it would be more tricky. You really do need that elaboration code to get called. And you have to be sure that the runtime is already present. And it may need elaboration too. And it may be wrong to re-elaborate parts of the runtime that have been elaborated by the already-running program. All this sounds like a very interesting project! One thought -- I'm not sure what the difference is between a .o and a .so, they are both ELF files (on Linux anyway), I wonder if dlopen etc would work with .o's? -S ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2003-12-30 16:56 UTC | newest] Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2003-12-18 23:31 load and use a ".o" file? lifetime n00b 2003-12-18 23:59 ` Stephen Leake 2003-12-19 1:51 ` tmoran 2003-12-19 15:28 ` lifetime n00b 2003-12-19 18:08 ` Stephen Leake 2003-12-20 20:12 ` lifetime n00b 2003-12-20 21:15 ` tmoran 2003-12-20 23:41 ` lifetime n00b 2003-12-21 7:15 ` tmoran 2003-12-21 11:46 ` Simon Wright 2003-12-21 13:57 ` Stephen Leake 2003-12-22 19:29 ` lifetime n00b 2003-12-22 20:49 ` Jon S. Anthony 2003-12-22 23:15 ` Stephen Leake 2003-12-23 1:36 ` tmoran 2003-12-27 22:55 ` Jon S. Anthony 2003-12-28 3:28 ` Stephen Leake 2003-12-28 16:14 ` Georg Bauhaus 2003-12-29 22:45 ` Jon S. Anthony 2003-12-29 22:42 ` Jon S. Anthony 2003-12-30 15:17 ` lifetime n00b 2003-12-30 16:56 ` Stephen Leake 2003-12-22 15:50 ` Mark H Johnson 2003-12-22 19:46 ` lifetime n00b 2003-12-22 22:58 ` Mark H Johnson 2003-12-23 17:48 ` Robert I. Eachus 2003-12-23 17:59 ` Mark H Johnson 2003-12-23 21:53 ` Robert I. Eachus 2003-12-19 21:28 ` Simon Wright
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox