Can shorten be made to shorten in unicode units rather than bytes?

Next topic

Author	Message
Sean Carney	Friday 06 February 2004 2:30:16 pm We are really happy with the Shorten function and do not mind that it cuts off words. But, we do have a problem where it cuts off unicode caracters in the middle and creates a garbage character. You can see an example at our page http://nsd.hopetalk.org We need to find a way to have shorten cut off based on unicode units.
Marco Zinn	Saturday 07 February 2004 11:46:09 am I'm not into unicode, but splitting the unicode character should not happen. I suggest, that you file a bug report. Marco http://www.hyperroad-design.com
Sean Carney	Sunday 22 February 2004 9:37:28 pm Thank you Marco. I filed a bug report. It also seems strange that shorten is cutting off some bytes even if the characters displayed are less then then characters that have been specified.
Jan Borsodi	Wednesday 03 March 2004 7:31:59 am PHP itself does not support Unicode internally. You can get some support with the mbstring extension and overriding internal text functions but not all of PHP will support it. We also use the mbstring extension (if available) to perform conversion when it's needed (instead of all the time). However our i18n system does not support text operation such as extraction a portion of it yet. This means that all template operators that modify text will not work on Unicode characters. The reason for the cutoff is the UTF8 encoding (which encodes Unicode characters), each Unicode character will be represented in an UTF8 encoding which can vary from 1 byte to 6 bytes. (1-3 is the most common). This means that a string that has three characters can actually be 4 or more bytes, and since PHP only sees each byte as a character it will cut off at the wrong place. The only way to get support for this is create all the various text operations that are being used in the operators and place them in the i18n library. Then change the operators to use that functionality. However this is not a small task, especially considering problems such as case mapping (lowercase, uppercase etc.). -- Amos Documentation: http://ez.no/ez_publish/documentation FAQ: http://ez.no/ez_publish/documentation/faq

eZ debug

Timing:	Jan 19 2025 01:00:13
Script start
Timing:	Jan 19 2025 01:00:13
Module start 'layout'
Timing:	Jan 19 2025 01:00:13
Module start 'content'
Timing:	Jan 19 2025 01:00:15
Module end 'content'
Timing:	Jan 19 2025 01:00:15
Script end

Main resources:

Total runtime	1.1597 sec
Peak memory usage	4,096.0000 KB
Database Queries	62

Timing points:

Checkpoint	Start (sec)	Duration (sec)	Memory at start (KB)	Memory used (KB)
Script start	0.0000	0.0053	590.5469	152.6563
Module start 'layout'	0.0053	0.0035	743.2031	39.5234
Module start 'content'	0.0089	1.1487	782.7266	588.0938
Module end 'content'	1.1575	0.0022	1,370.8203	12.4063
Script end	1.1597		1,383.2266

Time accumulators:

Accumulator	Duration (sec)	Duration (%)	Count	Average (sec)
Ini load
Load cache	0.0034	0.2943	16	0.0002
Check MTime	0.0016	0.1349	16	0.0001
Mysql Total
Database connection	0.0005	0.0413	1	0.0005
Mysqli_queries	1.1034	95.1390	62	0.0178
Looping result	0.0007	0.0638	60	0.0000
Template Total	1.1255	97.0	2	0.5627
Template load	0.0022	0.1889	2	0.0011
Template processing	1.1233	96.8553	2	0.5616
Template load and register function	0.0004	0.0322	1	0.0004
states
state_id_array	0.0018	0.1579	1	0.0018
state_identifier_array	0.0015	0.1259	2	0.0007
Override
Cache load	0.0018	0.1540	19	0.0001
Sytem overhead
Fetch class attribute can translate value	0.0015	0.1321	3	0.0005
Fetch class attribute name	0.0013	0.1095	6	0.0002
XML
Image XML parsing	0.0019	0.1643	3	0.0006
class_abstraction
Instantiating content class attribute	0.0000	0.0011	6	0.0000
General
dbfile	0.0020	0.1685	23	0.0001
String conversion	0.0000	0.0009	4	0.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

Usage	Requested template	Template	Template loaded
1	node/view/full.tpl	full/forum_topic.tpl	extension/sevenx/design/simple/override/templates/full/forum_topic.tpl
4	content/datatype/view/ezxmltext.tpl	<No override>	extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tpl
6	content/datatype/view/ezxmltags/paragraph.tpl	<No override>	extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tpl
2	content/datatype/view/ezimage.tpl	<No override>	extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tpl
3	content/datatype/view/ezxmltags/line.tpl	<No override>	design/standard/templates/content/datatype/view/ezxmltags/line.tpl
1	print_pagelayout.tpl	<No override>	extension/community/design/community/templates/print_pagelayout.tpl
Number of times templates used: 17 Number of unique templates used: 6

Time used to render debug report: 0.0001 secs