August 18, 2010 at 10:15 pm
I have a strange issue that is causing a headache. I have to output data from a view to a flat file. The data is to be comma separated and saved as txt.
The output must be capable of handling any character as data is entered from around the world.
While testing the output I found that foreign characters entered are not being written correctly. Further investigation revealed that if I just output one record with the character it is written correctly whereas any more than one and the character changes.
The character display on the web site correctly as UTF-8.
Various suggestions have been offered such as opening the file writing one record then closing. There are approx 500000 records so I suspect that is not a realistic solution.
I have tried incorporating data conversion but that also has not worked.
I am now desperate to find a solution.
Anyone any ideas
August 19, 2010 at 1:26 pm
There is a code page setting on the flat file connection manager. It is on the first page of the properties for the flat file. It should be set to 65001 for UTF-8.
Russ
Russel Loski, MCSE Business Intelligence, Data Platform
August 20, 2010 at 2:03 am
Thank you for that Russ but there is still a problem
Entered in the web site:- Its all greek to me ?
Results from query analyser:- Its all greek to me Î
Output to text file :-"Its all greek to me Î"
I have been told the requirement is to output the data as it is displayed on the screen.
Web front end uses UTF-8 coding
Checking another one of our front ends uses charset=iso-8859-1
and displays data as:- Its all greek to me Î
August 20, 2010 at 3:43 am
It looks to me like the problem is between the web site and the database, the first step in the process, not the SSIS part. Is the data being stored in a national character column (nchar, nvarchar, ntext) or a normal character column (char, varchar, text)? It should be stored in a national character column. If you have the database datatype right, the issue may be with the program that receives the input from the web site.
Russel Loski, MCSE Business Intelligence, Data Platform
August 20, 2010 at 6:08 am
Rus
I think the problem is with the data handling layer as the data is stored as nvarchar. It does appear that the appearance of data that is stored differs from what is displayed and it is not going to be possible to extract it the way they require.
Oh the trials and tribulations of database developement.
Dont you just love it when they say "well what do you suggest?" when you have exausted all avenues that you know of.
Many thanks for your help
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply