Ez Find don't index PDF with special chars like "ç"

Next topic

Author	Message
eric figo	Wednesday 19 November 2008 6:12:14 am Hi, I'm using EZ Find and pstotext in order to indexing PDF files. Some are indexed and some not. I tried many files and for exemple a pdf with this text single text is indexed : "Website un test d’indexation pour voir si ca marche ….. hsdhhdhhd" But if i change the "c" for à "ç" like "Website un test d’indexation pour voir si ça marche ….. hsdhhdhhd", the pdf is not indexed. Any ideas ? My database is in UTF 8, and i don't change the configurtaion of charset in EZ Publish. Thanks for your responses
Paul Borgermans	Wednesday 26 November 2008 10:59:20 am pstotext is not the best solution for converting pdf's to raw text, I guess that it fails to onvert the pdf file in question (try on the command line to se what happens) Better is to use pdftotext from the xpdf project, then configure a new script, for example called ezpdftotext with the following content (change the path tp pdftotext with your installation): #!/bin/sh <path to >/pdftotext -enc "UTF-8" $1 - And configure this script in binaryfile.ini Note that the default installation will "normalize" Latin1 characters, so eZ Find/Solr will transform "reçu" to "recu" and more ... so searching either form will produce the hit Best regards eZ Publish, eZ Find, Solr expert consulting and training http://twitter.com/paulborgermans
eric figo	Monday 01 December 2008 2:27:53 am HI, Thanks for the response. Some precisions, when I run pstotext in command line, with my pdf, I get the plain text without trouble. The problem is when i use the script to index, the files with specials chars are not index. I can't find them, even if I'm searching an over word ot the PDF without spécialchars. I tried you solution with pdftotext but I have the same problem. Best regards
Paul Borgermans	Friday 05 December 2008 1:17:21 pm Which versions are you using (ez find, ez publish)? eZ Publish, eZ Find, Solr expert consulting and training http://twitter.com/paulborgermans
eric figo	Tuesday 09 December 2008 1:09:54 am I'm using EZ Publish 4.0.1 with eZ Find 1.0.0beta2

eZ debug

Timing:	Jan 18 2025 10:40:48
Script start
Timing:	Jan 18 2025 10:40:48
Module start 'layout'
Timing:	Jan 18 2025 10:40:48
Module start 'content'
Timing:	Jan 18 2025 10:40:48
Module end 'content'
Timing:	Jan 18 2025 10:40:48
Script end

Main resources:

Total runtime	0.9217 sec
Peak memory usage	4,096.0000 KB
Database Queries	65

Timing points:

Checkpoint	Start (sec)	Duration (sec)	Memory at start (KB)	Memory used (KB)
Script start	0.0000	0.0095	589.6172	152.6563
Module start 'layout'	0.0095	0.0065	742.2734	39.5078
Module start 'content'	0.0160	0.9041	781.7813	561.1328
Module end 'content'	0.9201	0.0016	1,342.9141	12.1016
Script end	0.9216		1,355.0156

Time accumulators:

Accumulator	Duration (sec)	Duration (%)	Count	Average (sec)
Ini load
Load cache	0.0035	0.3771	16	0.0002
Check MTime	0.0013	0.1461	16	0.0001
Mysql Total
Database connection	0.0016	0.1786	1	0.0016
Mysqli_queries	0.8604	93.3513	65	0.0132
Looping result	0.0007	0.0726	63	0.0000
Template Total	0.8779	95.3	2	0.4390
Template load	0.0024	0.2601	2	0.0012
Template processing	0.8755	94.9931	2	0.4378
Template load and register function	0.0002	0.0201	1	0.0002
states
state_id_array	0.0010	0.1035	1	0.0010
state_identifier_array	0.0010	0.1088	2	0.0005
Override
Cache load	0.0019	0.2097	32	0.0001
Sytem overhead
Fetch class attribute can translate value	0.0011	0.1210	2	0.0006
Fetch class attribute name	0.0015	0.1677	6	0.0003
XML
Image XML parsing	0.0007	0.0717	2	0.0003
class_abstraction
Instantiating content class attribute	0.0000	0.0040	7	0.0000
General
dbfile	0.0008	0.0895	17	0.0000
String conversion	0.0000	0.0011	4	0.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

Usage	Requested template	Template	Template loaded
1	node/view/full.tpl	full/forum_topic.tpl	extension/sevenx/design/simple/override/templates/full/forum_topic.tpl
5	content/datatype/view/ezxmltext.tpl	<No override>	extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tpl
8	content/datatype/view/ezxmltags/paragraph.tpl	<No override>	extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tpl
3	content/datatype/view/ezxmltags/line.tpl	<No override>	design/standard/templates/content/datatype/view/ezxmltags/line.tpl
2	content/datatype/view/ezimage.tpl	<No override>	extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tpl
1	print_pagelayout.tpl	<No override>	extension/community/design/community/templates/print_pagelayout.tpl
Number of times templates used: 20 Number of unique templates used: 6

Time used to render debug report: 0.0001 secs