[Cuis] image segments
Luciano Notarfrancesco
luchiano at gmail.com
Thu Nov 26 09:31:24 CST 2015
Oh, yes, that's exactly what I wanted, awesome! Thanks!
On Thu, Nov 26, 2015 at 3:23 PM, H. Hirzel <hannes.hirzel at gmail.com> wrote:
> And this seems to cover what you want to do
>
> How to use an ImageSegment to determine memory usage
> http://wiki.squeak.org/squeak/1260
>
>
>
> On 11/26/15, H. Hirzel <hannes.hirzel at gmail.com> wrote:
> > That's interesting,
> >
> > see also http://wiki.squeak.org/squeak/2316 with a reference to an
> > article evaluation ImageSegments at the end.
> >
> >
> >
> > On 11/26/15, Luciano Notarfrancesco <luchiano at gmail.com> wrote:
> >> Cool, maybe it's possible to make it into a package, I'll take a look.
> >>
> >> I've never used it to store Projects, tho.. I used before to store
> >> things
> >> that must be loaded very fast on demand and can be forgotten once there
> >> are
> >> no more references to them.. like image fragments that compose a map of
> >> the
> >> world, and small sound fragments that are used to synthesize speech
> >> (diphones). Yesterday I just wanted to know how big are my objects, and
> I
> >> wanted to make some experiments to see how the bit-size of my objects
> >> grow
> >> as the problems I'm modeling grow by some other measure (degree of
> >> polynomials, size of matrices are the most basic) and to know how much
> >> memory are my algorithms actually using at every step... image segments
> >> are
> >> ideal for this, you can use them to know exactly how much memory would
> be
> >> freed if a reference to a given object were forgotten.
> >>
> >>
> >> On Thu, Nov 26, 2015 at 7:36 AM, H. Hirzel <hannes.hirzel at gmail.com>
> >> wrote:
> >>
> >>> Squeak 5.0 still has ImageSegments (class comment below). Saving and
> >>> loading of projects currently is blocked in Squeak because of an
> >>> unfinished name space (environments implementation).
> >>>
> >>> http://wiki.squeak.org/squeak/6218
> >>>
> >>> And there is Fuel
> >>> http://rmod.inria.fr/web/software/Fuel
> >>>
> >>> ----------
> >>>
> >>> ImageSegment
> >>>
> >>> I represent a segment of Squeak address space. I am created from an
> >>> array of root objects. After storing, my segment contains a binary
> >>> encoding of every object accessible from my roots but not otherwise
> >>> accessible from anywhere else in the system. My segment contains
> >>> outward pointers that are indices into my table of outPointers.
> >>> On load my segment is converted back into objects and becommed
> >>> into an Array of the loaded objects, so they can be enumerated.
> >>>
> >>> The main use of ImageSegments is to store Projects. A dummy
> >>> version of SmartRefStream traverses the Project. Everything it finds
> >>> is classified as either an object that is owned by the project (only
> >>> pointed to inside the project), or an object outside the project that
> >>> is pointed to from inside the project. The objects that are
> >>> completely owned by the project are compressed into pure binary form
> >>> in an ImageSegment. The outside objects are put in the 'outPointers'
> >>> array. The entire ImageSegment (binary part plus outPointers) is
> >>> encoded in a SmartRefStream, and saved on the disk. (aProject
> >>> exportSegmentWithChangeSet:fileName:directory:) calls (anImageSegment
> >>> writeForExportWithSources:inDirectory:changeSet:).
> >>>
> >>> Note that every object inside the project is put into the
> >>> segment's arrayOfRoots. This is because a dummy SmartRefStream to
> >>> scan the project, in order to make intelligent decisions about what
> >>> belongs in the project.
> >>> See Project's class comment for what messages are sent to
> >>> objects as they are unpacked in a new image.
> >>>
> >>> ---- Older Details ------
> >>>
> >>> The primary kind of image segment is an Export Segment. It
> >>> can be saved on a server and read into a completely different Squeak
> >>> image.
> >>> Old way to create one:
> >>> (ImageSegment new copyFromRootsForExport: (Array with: Baz with: Baz
> >>> class))
> >>> writeForExport: 'myFile.extSeg'.
> >>> Old way to create one for a project:
> >>> (Project named: 'Play With Me - 3') exportSegment.
> >>> To read it into another image: Select 'myFile.extSeg' in a FileList,
> >>> Menu 'load as project'. It will install its classes automatically.
> >>> If you need to see the roots array, it is temporarily stored in
> >>> (SmartRefStream scannedObject).
> >>>
> >>> Most of 'states' of an ImageSegment are not used to export a project,
> >>> and have been abandoned.
> >>>
> >>> When a segment is written out onto a file, it goes in a
> >>> folder called <image name>_segs. If your image is called
> >>> "Squeak2.6.image", the folder "Squeak2.6_segs" must accompany the
> >>> image whenever your move, copy, or rename it.
> >>> Whenever a Class is in arrayOfRoots, its class (aClass class)
> >>> must also be in the arrayOfRoots.
> >>> There are two kinds of image segments. Normal image segments
> >>> are a piece of a specific Squeak image, and can only be read back
> >>> into that image. The image holds the array of outPointers that are
> >>> necessary to turn the bits in the file into objects.
> >>> To put out a normal segment that holds a Project (not the
> >>> current project), execute (Project named: 'xxx') storeSegment.
> >>>
> >>>
> >>> arrayOfRoots The objects that head the tree we will trace.
> >>> segment The WordArray of raw bits of all objects in the
> >>> tree.
> >>> outPointers Oops of all objects outside the segment
> >>> pointed to from inside.
> >>> state (see below)
> >>> segmentName Its basic name. Often the name of a Project.
> >>> fileName The local name of the file. 'Foo-23.seg'
> >>> userRootCnt number of roots submitted by caller. Extras
> >>> are added in preparation for saving.
> >>>
> >>> state that an ImageSegment may exist in...
> >>>
> >>> #activeCopy (has been copied, with the intent to
> >>> become active)
> >>> arrayOfRoots, segment, and outPointers have been created by
> >>> copyFromRoots:. The tree of objects has been encoded in the segment,
> >>> but those objects are still present in the Squeak system.
> >>>
> >>> #active (segment is actively holding objects)
> >>> The segment is now the only holder of tree of objects. Each of the
> >>> original roots has been transmuted into an ImageSegmentRootStub that
> >>> refers back to this image segment. The original objects in the
> >>> segment will all be garbageCollected.
> >>>
> >>> #onFile
> >>> The segment has been written out to a file and replaced by a file
> >>> pointer. Only ImageSegmentRootStubs and the array of outPointers
> >>> remains in the image. To get this far:
> >>> (ImageSegment new copyFromRoots: (Array with: Baz with: Baz class))
> >>> writeToFile: 'myFile.seg'.
> >>>
> >>> #inactive
> >>> The segment has been brought back into memory and turned back into
> >>> objects. rootsArray is set, but the segment is invalid.
> >>>
> >>> #onFileWithSymbols
> >>> The segment has been written out to a file, along with the text of
> >>> all the symbols in the outPointers array, and replaced by a file
> >>> pointer. This reduces the size of the outPointers array, and also
> >>> allows the system to reclaim any symbols that are not referred to
> >>> from elsewhere in the image. The specific format used is that of a
> >>> literal array as follows:
> >>> #(symbol1 symbol2 # symbol3 symbol4 'symbolWithSpaces' #
> >>> symbol5).
> >>> In this case, the original outPointers array was 8 long, but the
> >>> compacted table of outPointers retains only two entries. These get
> >>> inserted in place of the #'s in the array of symbols after it is read
> >>> back in. Symbols with embedded spaces or other strange characters
> >>> are written as strings, and converted back to symbols when read back
> >>> in. The symbol # is never written out.
> >>> NOTE: All IdentitySets or dictionaries must be rehashed when
> >>> being read back from this format. The symbols are effectively
> >>> internal. (No, not if read back into same image. If a different
> >>> image, then use #imported. -tk)
> >>>
> >>> #imported
> >>> The segment is on an external file or just read in from one. The
> >>> segment and outPointers are meant to be read into a foreign image.
> >>> In this form, the image segment can be read from a URL, and
> >>> installed. A copy of the original array of root objects is
> >>> constructed, with former outPointers bound to existing objects in the
> >>> host system.
> >>> (Any Class inside the segment MUST be in the arrayOfRoots.
> >>> This is so its association can be inserted into Smalltalk. The
> >>> class's metaclass must be in roots also. Methods that are in
> >>> outPointers because blocks point at them, were found and added to the
> >>> roots.
> >>> All IdentitySets and dictionaries are rehashed when being
> >>> read back from exported segments.)
> >>>
> >>>
> >>> To discover why only some of the objects in a project are being
> >>> written out, try this (***Destructive Test***). This breaks lots of
> >>> backpointers in the target project, and puts up an array of
> >>> suspicious objects, a list of the classes of the outPointers, and a
> >>> debugger.
> >>> "Close any transcripts in the target project"
> >>> World currentHand objectToPaste ifNotNil: [
> >>> self inform: 'Hand is holding a Morph in its paste buffer:\'
> >>> withCRs,
> >>> World currentHand objectToPaste printString].
> >>> PV := Project named: 'xxxx'.
> >>> (IS := ImageSegment new) findRogueRootsImSeg:
> >>> (Array with: PV world presenter with: PV world).
> >>> IS findOwnersOutPtrs. "Optionally: write a file with owner chains"
> >>> "Quit and DO NOT save"
> >>>
> >>> When an export image segment is brought into an image, it is like an
> >>> image starting up. Certain startUp messages need to be run. These
> >>> are byte and word reversals for nonPointer data that comes from a
> >>> machine of the opposite endianness. #startUpProc passes over all
> >>> objects in the segment, and:
> >>> The first time an instance of class X is encountered, (msg _
> >>> X startUpFrom: anImageSegment) is sent. If msg is nil, the usual
> >>> case, it means that instances of X do not need special work. X is
> >>> included in the IdentitySet, noStartUpNeeded. If msg is not nil,
> >>> store it in the dictionary, startUps (aClass -> aMessage).
> >>> When a later instance of X is encountered, if X is in
> >>> noStartUpNeeded, do nothing. If X is in startUps, send the message
> >>> to the instance. Typically this is a message like #swapShortObjects.
> >>> Every class that implements #startUp, should see if it needs
> >>> a parallel implementation of #startUpFrom:.
> >>>
> >>> On 11/26/15, Luciano Notarfrancesco <luchiano at gmail.com> wrote:
> >>> > I don't see image segments in Cuis... I don't even know if they still
> >>> exist
> >>> > in Squeak... Anyone is still using it?
> >>> >
> >>> > It would be cool to have it at least as a package. If I remember
> >>> correctly
> >>> > it was very simple, like two primitives to load and save, but I think
> >>> > it
> >>> > depended on something like mark and sweep or some particular kind of
> >>> > garbage collector and it might not work in new VMs.
> >>> >
> >>>
> >>> _______________________________________________
> >>> Cuis mailing list
> >>> Cuis at jvuletich.org
> >>> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
> >>>
> >>
> >
>
> _______________________________________________
> Cuis mailing list
> Cuis at jvuletich.org
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jvuletich.org/pipermail/cuis_jvuletich.org/attachments/20151126/36cbae51/attachment-0004.html>
More information about the Cuis
mailing list