Comments for My IT Weblog http://scorreiait.wordpress.com Just another WordPress.com weblog Mon, 31 Aug 2009 16:12:30 +0000 http://wordpress.com/ hourly 1 Comment on Using regular expressions when profiling SQL Server with Talend Open Profiler by Amine http://scorreiait.wordpress.com/2009/08/28/using-regular-expressions-when-profiling-sql-server-with-talend-open-profiler/#comment-155 Amine Mon, 31 Aug 2009 16:12:30 +0000 http://scorreiait.wordpress.com/?p=196#comment-155 Thanks Sebastiao , If any one needs more help to install this or others functions , be free to contact me. hallam-dz.com Thanks Sebastiao ,

If any one needs more help to install this or others functions , be free to contact me.
hallam-dz.com

]]> Comment on How to detect random text in a free text field? by scorreia http://scorreiait.wordpress.com/2009/03/13/how-to-detect-random-text-in-a-free-text-field/#comment-145 scorreia Thu, 02 Jul 2009 18:17:12 +0000 http://scorreiait.wordpress.com/?p=171#comment-145 yes, the physical proximity of the keys is what gave me this idea. But it's also because on the French keyboard, there is no vowel on the home row. Hence the words created with the keys of the home row only have a low probability to be valid words. I have run some tests on real data with emails and company names. I have found a few percentage of invalid data. For example, I have found the following invalid emails: szdfsdf@dfsdf.sdfsd gddggh@nfngn.fhgfh bnmbnm@gffdg.grgtrg cdcdc@vde.dcdcw fdghfdgh@fghfg.dfghfd ff@fgfh.gbvgg adsdc@dscsdcdsc.cddcsc adsdc@dscsdcdsc.cddcsc rhhtyhtyhtyh@rthgrthr.frtrgtr vbcvb@cvbcvb.cvbcv sdf@adf.pldff ggugiug@ghghg.ckcjf hkl@domain.sdfgsdfg and for the companies: fghfg ssdfgfsdgvgx SDCDSC dslck dscsd sdcfscds fdvdfv xcvxcvx dfsdf sqsqsq xvxvxv sdfsdf sddssdsdsd cqwxd ANPCyT gdfgdf fffhk drgdrg drgdrg hjhkjhkj hbhbjhb hbhbjhb nttcw sgdfg BCBSF asdgggs ytyrtytttttttttttttttttttttt qwdqw dddqd sdfsdfsdfsdf gfjhfg In fact, I had to change my regular expression and search for 5 consecutive consonants instead of 4. The reason is that some companies have 4 consecutive consonants. For example: SNCF and many german companies contain the acronym GmbH. But when I used my smaller regular expression limited to the home row keys, these terms do not match. yes, the physical proximity of the keys is what gave me this idea. But it’s also because on the French keyboard, there is no vowel on the home row. Hence the words created with the keys of the home row only have a low probability to be valid words.

I have run some tests on real data with emails and company names. I have found a few percentage of invalid data.

For example, I have found the following invalid emails:
szdfsdf@dfsdf.sdfsd
gddggh@nfngn.fhgfh
bnmbnm@gffdg.grgtrg
cdcdc@vde.dcdcw
fdghfdgh@fghfg.dfghfd
ff@fgfh.gbvgg
adsdc@dscsdcdsc.cddcsc
adsdc@dscsdcdsc.cddcsc
rhhtyhtyhtyh@rthgrthr.frtrgtr
vbcvb@cvbcvb.cvbcv
sdf@adf.pldff
ggugiug@ghghg.ckcjf
hkl@domain.sdfgsdfg

and for the companies:
fghfg
ssdfgfsdgvgx
SDCDSC
dslck
dscsd
sdcfscds
fdvdfv
xcvxcvx
dfsdf
sqsqsq
xvxvxv
sdfsdf
sddssdsdsd
cqwxd
ANPCyT
gdfgdf
fffhk
drgdrg
drgdrg
hjhkjhkj
hbhbjhb
hbhbjhb
nttcw
sgdfg
BCBSF
asdgggs
ytyrtytttttttttttttttttttttt
qwdqw
dddqd
sdfsdfsdfsdf
gfjhfg

In fact, I had to change my regular expression and search for 5 consecutive consonants instead of 4. The reason is that some companies have 4 consecutive consonants. For example: SNCF and many german companies contain the acronym GmbH.

But when I used my smaller regular expression limited to the home row keys, these terms do not match.

]]> Comment on How to detect random text in a free text field? by bronius http://scorreiait.wordpress.com/2009/03/13/how-to-detect-random-text-in-a-free-text-field/#comment-144 bronius Wed, 01 Jul 2009 19:53:28 +0000 http://scorreiait.wordpress.com/?p=171#comment-144 Ha! Clever idea. The general concept is "detecting a number of characters entered whose keys are in physical proximity of one-another". I like it. How is it working out for you in practice? Ha! Clever idea. The general concept is “detecting a number of characters entered whose keys are in physical proximity of one-another”.

I like it. How is it working out for you in practice?

]]> Comment on How to detect random text in a free text field? by Datenqualität in Textfeldern mit RegExp überprüfen - dijit http://scorreiait.wordpress.com/2009/03/13/how-to-detect-random-text-in-a-free-text-field/#comment-112 Datenqualität in Textfeldern mit RegExp überprüfen - dijit Fri, 13 Mar 2009 18:09:09 +0000 http://scorreiait.wordpress.com/?p=171#comment-112 [...] Ansatz, um Texteingaben nach bewussten Falscheingaben zu durchsuchen, hat mein Kollege Sebastiao in seinem Blog veröffentlicht.Er macht sich dabei eine sehr interessante Tatsache zu nutze - das [...] [...] Ansatz, um Texteingaben nach bewussten Falscheingaben zu durchsuchen, hat mein Kollege Sebastiao in seinem Blog veröffentlicht.Er macht sich dabei eine sehr interessante Tatsache zu nutze – das [...]

]]> Comment on Talend Open Studio vs Pentaho Data Integration by scorreia http://scorreiait.wordpress.com/2008/03/29/talend-open-studio-vs-pentaho-data-integration/#comment-109 scorreia Sun, 01 Mar 2009 13:19:32 +0000 http://scorreiait.wordpress.com/?p=30#comment-109 Hi, sorry for this late reply, I missed your last comment. I don't know about a document related to Cognos. Hi,

sorry for this late reply, I missed your last comment. I don’t know about a document related to Cognos.

]]> Comment on Talend Open Studio vs Pentaho Data Integration by Pawan http://scorreiait.wordpress.com/2008/03/29/talend-open-studio-vs-pentaho-data-integration/#comment-102 Pawan Mon, 02 Feb 2009 05:21:07 +0000 http://scorreiait.wordpress.com/?p=30#comment-102 Thanks a lot for this nice benchmarking doc. Do you have any benchmarking doc related to "Cognos Vs Japser or Pentaho". Thanks a lot for this nice benchmarking doc. Do you have any benchmarking doc related to “Cognos Vs Japser or Pentaho”.

]]> Comment on Talend Open Studio vs Pentaho Data Integration by scorreia http://scorreiait.wordpress.com/2008/03/29/talend-open-studio-vs-pentaho-data-integration/#comment-101 scorreia Fri, 30 Jan 2009 15:35:41 +0000 http://scorreiait.wordpress.com/?p=30#comment-101 Hi, I don't know about an English version of this paper. But you may find <a href="http://marcrussel.files.wordpress.com/2008/10/etlbenchmarks_manappsc221008.pdf" rel="nofollow">this benchmark</a> interesting. It is commented on <a href="http://marcrussel.wordpress.com/2008/10/29/etl-benchmark-by-manapps/" rel="nofollow">Marc Russel's blog</a> and on <a href="http://blog.gobansaor.com/2008/10/30/open-source-metrics/" rel="nofollow">Goban Saor's blog</a>. Hi,

I don’t know about an English version of this paper. But you may find this benchmark interesting. It is commented on Marc Russel’s blog and on Goban Saor’s blog.

]]> Comment on Talend Open Studio vs Pentaho Data Integration by Pawan http://scorreiait.wordpress.com/2008/03/29/talend-open-studio-vs-pentaho-data-integration/#comment-100 Pawan Fri, 30 Jan 2009 07:19:54 +0000 http://scorreiait.wordpress.com/?p=30#comment-100 Hi, I am really interested in checking this paper but, i don't know french. Does anybody have english version of this paper? Hi, I am really interested in checking this paper but, i don’t know french. Does anybody have english version of this paper?

]]> Comment on Talend Open Profiler by Cécile Maindron http://scorreiait.wordpress.com/2008/06/20/talend-open-profiler/#comment-40 Cécile Maindron Fri, 01 Aug 2008 11:58:47 +0000 http://scorreiait.wordpress.com/?p=38#comment-40 Do you want to learn more about Talend Open Profiler? Are you interested in seeing a live demo of the tool? Reserve your Webinar seat now at: https://www1.gotomeeting.com/register/472014645 (English version) https://www1.gotomeeting.com/register/256033456 (French version) https://www1.gotomeeting.com/register/553490277 (German version) To view a full calendar of Talend Webinars: http://www.talend.com/campaign/campaign.php?id=125&src=PostCalendarWebinar Do you want to learn more about Talend Open Profiler? Are you interested in seeing a live demo of the tool? Reserve your Webinar seat now at: https://www1.gotomeeting.com/register/472014645 (English version)
https://www1.gotomeeting.com/register/256033456 (French version)
https://www1.gotomeeting.com/register/553490277 (German version)

To view a full calendar of Talend Webinars:
http://www.talend.com/campaign/campaign.php?id=125&src=PostCalendarWebinar

]]> Comment on Talend Open Profiler by Database Management » Blog Archive » Talend Open Profiler http://scorreiait.wordpress.com/2008/06/20/talend-open-profiler/#comment-38 Database Management » Blog Archive » Talend Open Profiler Fri, 20 Jun 2008 22:31:12 +0000 http://scorreiait.wordpress.com/?p=38#comment-38 [...] creativelinkjobs.com wrote an interesting post today onHere’s a quick excerpt I have been working on this project for a few months now and I am pleased to announce the first public release candidate of Talend Open Profiler. This tool helps you to browse, explore your databases and analyze your data. For each column that you want to analyze, you have several indicators at your disposal: row counts, null counts, duplicate counts… field length, frequency table, summary statistics… There are also indicators based on regular expressions. These indicators helps you to discov [...] [...] creativelinkjobs.com wrote an interesting post today onHere’s a quick excerpt I have been working on this project for a few months now and I am pleased to announce the first public release candidate of Talend Open Profiler. This tool helps you to browse, explore your databases and analyze your data. For each column that you want to analyze, you have several indicators at your disposal: row counts, null counts, duplicate counts… field length, frequency table, summary statistics… There are also indicators based on regular expressions. These indicators helps you to discov [...]

]]>