[Cuis] image segments
Juan Vuletich
juan at jvuletich.org
Sat Nov 28 14:20:17 CST 2015
At some point in the development of Cuis I removed both Projects and
ImageSegments. It shouldn't be hard to add them back, as an optional
package.
WRT Fuel, it would be a great addition. If (as I believe) it already
works on Squeak, besides Pharo, then it shouldn't be too hard to port to
Cuis.
Cheers,
Juan Vuletich
On 26/11/2015 11:10 a.m., Luciano Notarfrancesco wrote:
> Cool, maybe it's possible to make it into a package, I'll take a look.
>
> I've never used it to store Projects, tho.. I used before to store
> things that must be loaded very fast on demand and can be forgotten
> once there are no more references to them.. like image fragments that
> compose a map of the world, and small sound fragments that are used to
> synthesize speech (diphones). Yesterday I just wanted to know how big
> are my objects, and I wanted to make some experiments to see how the
> bit-size of my objects grow as the problems I'm modeling grow by some
> other measure (degree of polynomials, size of matrices are the most
> basic) and to know how much memory are my algorithms actually using at
> every step... image segments are ideal for this, you can use them to
> know exactly how much memory would be freed if a reference to a given
> object were forgotten.
>
>
> On Thu, Nov 26, 2015 at 7:36 AM, H. Hirzel <hannes.hirzel at gmail.com
> <mailto:hannes.hirzel at gmail.com>> wrote:
>
> Squeak 5.0 still has ImageSegments (class comment below). Saving and
> loading of projects currently is blocked in Squeak because of an
> unfinished name space (environments implementation).
>
> http://wiki.squeak.org/squeak/6218
>
> And there is Fuel
> http://rmod.inria.fr/web/software/Fuel
>
> ----------
>
> ImageSegment
>
> I represent a segment of Squeak address space. I am created from an
> array of root objects. After storing, my segment contains a binary
> encoding of every object accessible from my roots but not otherwise
> accessible from anywhere else in the system. My segment contains
> outward pointers that are indices into my table of outPointers.
> On load my segment is converted back into objects and becommed
> into an Array of the loaded objects, so they can be enumerated.
>
> The main use of ImageSegments is to store Projects. A dummy
> version of SmartRefStream traverses the Project. Everything it finds
> is classified as either an object that is owned by the project (only
> pointed to inside the project), or an object outside the project that
> is pointed to from inside the project. The objects that are
> completely owned by the project are compressed into pure binary form
> in an ImageSegment. The outside objects are put in the 'outPointers'
> array. The entire ImageSegment (binary part plus outPointers) is
> encoded in a SmartRefStream, and saved on the disk. (aProject
> exportSegmentWithChangeSet:fileName:directory:) calls (anImageSegment
> writeForExportWithSources:inDirectory:changeSet:).
>
> Note that every object inside the project is put into the
> segment's arrayOfRoots. This is because a dummy SmartRefStream to
> scan the project, in order to make intelligent decisions about what
> belongs in the project.
> See Project's class comment for what messages are sent to
> objects as they are unpacked in a new image.
>
> ---- Older Details ------
>
> The primary kind of image segment is an Export Segment. It
> can be saved on a server and read into a completely different Squeak
> image.
> Old way to create one:
> (ImageSegment new copyFromRootsForExport: (Array with: Baz with:
> Baz class))
> writeForExport: 'myFile.extSeg'.
> Old way to create one for a project:
> (Project named: 'Play With Me - 3') exportSegment.
> To read it into another image: Select 'myFile.extSeg' in a FileList,
> Menu 'load as project'. It will install its classes automatically.
> If you need to see the roots array, it is temporarily stored in
> (SmartRefStream scannedObject).
>
> Most of 'states' of an ImageSegment are not used to export a project,
> and have been abandoned.
>
> When a segment is written out onto a file, it goes in a
> folder called <image name>_segs. If your image is called
> "Squeak2.6.image", the folder "Squeak2.6_segs" must accompany the
> image whenever your move, copy, or rename it.
> Whenever a Class is in arrayOfRoots, its class (aClass class)
> must also be in the arrayOfRoots.
> There are two kinds of image segments. Normal image segments
> are a piece of a specific Squeak image, and can only be read back
> into that image. The image holds the array of outPointers that are
> necessary to turn the bits in the file into objects.
> To put out a normal segment that holds a Project (not the
> current project), execute (Project named: 'xxx') storeSegment.
>
>
> arrayOfRoots The objects that head the tree we will trace.
> segment The WordArray of raw bits of all objects
> in the tree.
> outPointers Oops of all objects outside the segment
> pointed to from inside.
> state (see below)
> segmentName Its basic name. Often the name of a Project.
> fileName The local name of the file. 'Foo-23.seg'
> userRootCnt number of roots submitted by caller. Extras
> are added in preparation for saving.
>
> state that an ImageSegment may exist in...
>
> #activeCopy (has been copied, with the intent to
> become active)
> arrayOfRoots, segment, and outPointers have been created by
> copyFromRoots:. The tree of objects has been encoded in the segment,
> but those objects are still present in the Squeak system.
>
> #active (segment is actively holding objects)
> The segment is now the only holder of tree of objects. Each of the
> original roots has been transmuted into an ImageSegmentRootStub that
> refers back to this image segment. The original objects in the
> segment will all be garbageCollected.
>
> #onFile
> The segment has been written out to a file and replaced by a file
> pointer. Only ImageSegmentRootStubs and the array of outPointers
> remains in the image. To get this far:
> (ImageSegment new copyFromRoots: (Array with: Baz with: Baz class))
> writeToFile: 'myFile.seg'.
>
> #inactive
> The segment has been brought back into memory and turned back into
> objects. rootsArray is set, but the segment is invalid.
>
> #onFileWithSymbols
> The segment has been written out to a file, along with the text of
> all the symbols in the outPointers array, and replaced by a file
> pointer. This reduces the size of the outPointers array, and also
> allows the system to reclaim any symbols that are not referred to
> from elsewhere in the image. The specific format used is that of a
> literal array as follows:
> #(symbol1 symbol2 # symbol3 symbol4 'symbolWithSpaces' #
> symbol5).
> In this case, the original outPointers array was 8 long, but the
> compacted table of outPointers retains only two entries. These get
> inserted in place of the #'s in the array of symbols after it is read
> back in. Symbols with embedded spaces or other strange characters
> are written as strings, and converted back to symbols when read back
> in. The symbol # is never written out.
> NOTE: All IdentitySets or dictionaries must be rehashed when
> being read back from this format. The symbols are effectively
> internal. (No, not if read back into same image. If a different
> image, then use #imported. -tk)
>
> #imported
> The segment is on an external file or just read in from one. The
> segment and outPointers are meant to be read into a foreign image.
> In this form, the image segment can be read from a URL, and
> installed. A copy of the original array of root objects is
> constructed, with former outPointers bound to existing objects in the
> host system.
> (Any Class inside the segment MUST be in the arrayOfRoots.
> This is so its association can be inserted into Smalltalk. The
> class's metaclass must be in roots also. Methods that are in
> outPointers because blocks point at them, were found and added to the
> roots.
> All IdentitySets and dictionaries are rehashed when being
> read back from exported segments.)
>
>
> To discover why only some of the objects in a project are being
> written out, try this (***Destructive Test***). This breaks lots of
> backpointers in the target project, and puts up an array of
> suspicious objects, a list of the classes of the outPointers, and a
> debugger.
> "Close any transcripts in the target project"
> World currentHand objectToPaste ifNotNil: [
> self inform: 'Hand is holding a Morph in its paste
> buffer:\' withCRs,
> World currentHand objectToPaste printString].
> PV := Project named: 'xxxx'.
> (IS := ImageSegment new) findRogueRootsImSeg:
> (Array with: PV world presenter with: PV world).
> IS findOwnersOutPtrs. "Optionally: write a file with owner chains"
> "Quit and DO NOT save"
>
> When an export image segment is brought into an image, it is like an
> image starting up. Certain startUp messages need to be run. These
> are byte and word reversals for nonPointer data that comes from a
> machine of the opposite endianness. #startUpProc passes over all
> objects in the segment, and:
> The first time an instance of class X is encountered, (msg _
> X startUpFrom: anImageSegment) is sent. If msg is nil, the usual
> case, it means that instances of X do not need special work. X is
> included in the IdentitySet, noStartUpNeeded. If msg is not nil,
> store it in the dictionary, startUps (aClass -> aMessage).
> When a later instance of X is encountered, if X is in
> noStartUpNeeded, do nothing. If X is in startUps, send the message
> to the instance. Typically this is a message like #swapShortObjects.
> Every class that implements #startUp, should see if it needs
> a parallel implementation of #startUpFrom:.
>
> On 11/26/15, Luciano Notarfrancesco <luchiano at gmail.com
> <mailto:luchiano at gmail.com>> wrote:
> > I don't see image segments in Cuis... I don't even know if they
> still exist
> > in Squeak... Anyone is still using it?
> >
> > It would be cool to have it at least as a package. If I remember
> correctly
> > it was very simple, like two primitives to load and save, but I
> think it
> > depended on something like mark and sweep or some particular kind of
> > garbage collector and it might not work in new VMs.
> >
>
> _______________________________________________
> Cuis mailing list
> Cuis at jvuletich.org <mailto:Cuis at jvuletich.org>
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>
>
>
> _______________________________________________
> Cuis mailing list
> Cuis at jvuletich.org
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jvuletich.org/pipermail/cuis_jvuletich.org/attachments/20151128/5e6e6890/attachment-0004.html>
More information about the Cuis
mailing list