[Cuis] (Minimal) requirements for Unicode support?

[H. Hirzel]

Hello Juan

On 2/8/13, Juan Vuletich <juan at jvuletich.org> wrote:
> Hi Hannes, Folks,
>
> I agree with all you say, and I started working a bit on the needed
> support in base Cuis.

> For your points 1) and 2), I published a few updates. You recently
> implemented NCRs (i.e. like 𞃳) when converting UTF-8 to String.
> I added hex besides decimal, and most important, loss less conversion
> back to UTF-8.

Good. I have seen the parsing in the new method
String>>asUtf8: convertEmbeddedNCRs


> This would allow editing any UTF-8 text file, or copy/paste to external
> editors (without being able to actually see any char outside Latin, of
> course).

Yes, this indeed will expand the area of application possibilities for Cuis.


>I didn't hook it yet into the system. It should also be enough
> to serve any Unicode content in web servers such as Aida.

Yes,
If you google for
   statistics character encoding  html

the first hit claims that 75% of all web pages are encoded as UTF8.




> I also enhanced a bit the style, so I moved and renamed methods. I also
> made UTF-8 to be stored in ByteArrays and not Strings. This opens the
> path to alternative String representations, and makes more sense to me.

String is
   ArrayedCollection variableByteSubclass: #String

and ByteArray is a sibling class
   ArrayedCollection variableByteSubclass: #ByteArray

That is fine.

> Unfortunately, this means I broke the examples at
> https://github.com/hhzl/Cuis-Add-Ons/blob/master/UnicodeNotes.md . I
> apologize for that.

I do not mind :-)   I'll update them.

> Please take a look.

I did and will do more.

The next question is now how to hook these Unicode utility methods
into the system.

Thank you for the updates in Cuis.

Regards

--Hannes


> On 2/7/2013 12:01 PM, H. Hirzel wrote:
>> Hello Juan, Angel, Janko, Germán, Kean and others.
>>
>> Thank your for sharing ideas and engaging in this discussion.
>>
>> As we are discussing now a possible Unicode support on the conceptual
>> level the question arises:
>>
>>      What are the minimal requirements for Unicode support for Cuis?
>>
>> Let me state how I see it: (perception form outside , no implementation
>> issues)
>>
>>
>> ## First of all surely
>>
>>
>> Support of reading and writing of UFT8 encoded files.
>> To a certain extent this is the case as of now. But is should be improved.
>>
>> This may or may not mean to drop ISO8859-15 completely as file encoding
>> format.
>>
>>
>>
>>
>>
>> ## Secondly
>>
>> It should be possible to port web libraries like Swazoo, WebClient,
>> Aida, Zinc, Altitude and others without problems. This does not
>> necessarily mean that the files appear 'nicely' in Cuis.
>>
>>
>> ## Thirdly
>>
>> Unicode support for display in Cuis. Probably still the major European
>> languages plus special symbols. More or less the state as is plus a
>> few more symbols/
>>
>>
>> ##  Fourthly
>>
>> Having a foundation that people who want to add additional support
>> (e.g. Korean, Russian, Japanese) can go ahead and implement
>> add-on-libraries.
>>
>>
>>
>> ##  Focus
>>
>> We should focus on 1) and 2) first.
>>
>> As of now I think I am on a good way to support this with an external
>> library. As for the changes in Cuis as such I think we should be
>> careful not to add too much complexity.
>> I prefer Cuis to remain lean.
>>
>>
>> ## CONCLUSION
>>
>> Let's us come to an agreement
>>
>> a) what the minimal support for Unicode should be.
>>
>> b) what is the task of Juan to add into Cuis and what should reside in
>> external libraries.
>>
>>
>> In the meantime I'll update the notes
>>
>>     https://github.com/hhzl/Cuis-Add-Ons/blob/master/UnicodeNotes.md
>>
>> here describing what the current Unicode support is.
>>
>>
>> Kind regards
>> Hannes
>>
>> _______________________________________________
>> Cuis mailing list
>> Cuis at jvuletich.org
>> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>>
>>
>> -----
>> Se certificó que el correo no contiene virus.
>> Comprobada por AVG - www.avg.es
>> Versión: 2013.0.2897 / Base de datos de virus: 2639/6087 - Fecha de la
>> versión: 07/02/2013
>
>
> _______________________________________________
> Cuis mailing list
> Cuis at jvuletich.org
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>