Sample Header Ad - 728x90

Database Administrators

Q&A for database professionals who wish to improve their database skills

Latest Questions

0 votes
0 answers
19 views
SSIS Data Quality Services Issue - Object reference not set to an instance of an object
I have tried to load data using my SSIS pipelines and there is no issue when the source table have around 90k+ data. This Data Quality Services component works well. But once the data has reached 100k+ (132,812) rows, it suddenly breaking and produce this error: > [DQS Cleansing] Error: An unexpecte...
I have tried to load data using my SSIS pipelines and there is no issue when the source table have around 90k+ data. This Data Quality Services component works well. But once the data has reached 100k+ (132,812) rows, it suddenly breaking and produce this error: > [DQS Cleansing] Error: An unexpected error occurred in the DQS infrastructure. System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.Ssdqs.Component.DataCorrection.Logic.DataCorrectionComponent.ProcessChunk(ReadOnlyCollection1 fieldMappings, ReadOnlyCollection1 records, CorrectedRecordsStatusStatistics& correctedRecordsTotalStatusStatistics) > [DQS Cleansing ] Error: Microsoft.Ssdqs.Infra.Exceptions.EntryPointException: The attempt to update or delete a DAO object of type 'AKnowledgebase' with id 1000175 has failed because the object is not up to date or is being deleted from the database. at Microsoft.Ssdqs.Proxy.Database.DBAccessClient.Exec() at Microsoft.Ssdqs.Proxy.EntryPoint.KnowledgebaseManagementEntryPointClient.DQProjectDelete(DQProject dqProject) at Microsoft.Ssdqs.Component.DataCorrection.Logic.DataCorrectionComponent.ProcessInput(Int32 inputID, PipelineBuffer buffer) at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket) > [SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "DQS Cleansing" (369) failed with error code 0x80131500 while processing input "DQS Cleansing Input" (382). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure. This is my current pipeline that I come up with: enter image description here And this is the same pipeline but after the 3rd load I made: enter image description here **I have tried googling** this error here but it does not help resolve the error. Is there anyway I can fix this issue? I will also appreciate any resource if you can provide.
Amir Hamzah (11 rep)
Jul 16, 2025, 01:34 AM • Last activity: Jul 16, 2025, 02:12 AM
5 votes
2 answers
364 views
How can we trim bad data and output clean trimmed data using DQS or MDS?
Until now, we have used stored procedures for data massaging and cleansing but we have now decided to move to using Data Quality Services (DQS) and Master Data Management (MDS) in our data warehouse projects. There is one very common function which we use in our stored procedures - it trims and repl...
Until now, we have used stored procedures for data massaging and cleansing but we have now decided to move to using Data Quality Services (DQS) and Master Data Management (MDS) in our data warehouse projects. There is one very common function which we use in our stored procedures - it trims and replaces the , character with white space e.g.: UPDATE [Table] SET Column=(ltrim(rtrim(replace(Column,',','')))) I have spent hours to figure out a way to achieve this using DQS and MDM but couldn't find any way to achieve the same goal. I would really appreciate if you guys can help me out with this. In fact, thank you already for your precious time.
Ghalib Mustafa (51 rep)
Apr 30, 2019, 02:39 PM • Last activity: Jun 19, 2019, 08:20 AM
4 votes
1 answers
2728 views
How to remove Data Quality Services completely?
When I setup one of our SQLServer 2012 instances a while back, I didn't realize that I have checked "Data Quality Services" feature. Looking at my installation notes, I find that I didn't end up performing any of the post-install task to install this feature so we didn't end up using this feature. H...
When I setup one of our SQLServer 2012 instances a while back, I didn't realize that I have checked "Data Quality Services" feature. Looking at my installation notes, I find that I didn't end up performing any of the post-install task to install this feature so we didn't end up using this feature. However, when I run the "Installed SQL Server features discovery report", the report would shows that this feature is configured, even though we don't have any of the DQS databases and any of its objects loaded in the instance. Can someone tell me how to completely remove this feature so that it doesn't show up anymore in the Microsoft SQL Server 2012 Setup Discovery Report? As always, your help is greatly appreciated.
Schmallen (129 rep)
Oct 2, 2017, 11:51 PM • Last activity: May 21, 2018, 09:54 PM
0 votes
1 answers
586 views
MS SQL Server 2012 Data Quality Services (DQS) installation failure
I tried to install MS SQL Server 2012 DQS using DQSInstaller.exe, but installation failed. In the console output started by the install tool (DQSInstaller.exe) I found this line: Password validation failed. The password does not meet Windows policy requirements because it is too short. It was a bit...
I tried to install MS SQL Server 2012 DQS using DQSInstaller.exe, but installation failed. In the console output started by the install tool (DQSInstaller.exe) I found this line: Password validation failed. The password does not meet Windows policy requirements because it is too short. It was a bit odd, since my password was 15 characters long, but who knows. I used a different one and made it sure that it is 30 chars long. That didn't help and the installer insisted that the password is still too short. Next thing was upgrading my SQL Server 2012 to SP1. That didn't help either. The Installer is started with admin privileges. The database instance is NOT a default instance, but I see the script finds that out by itself. What could be the problem?
OzrenTkalcecKrznaric (250 rep)
Jan 21, 2016, 08:43 AM • Last activity: Aug 18, 2017, 06:53 AM
1 votes
0 answers
10 views
SSDT no avaible input column error
i am learning SSDT from this link https://msdn.microsoft.com/en-us/library/jj819740(v=sql.120).aspx on point 8 i stuck before that i install DQS successfully and also connected to Data Quality service client now when i check in MS SQL 2014 there is 3 database created DQS_MAIN,DQS_PROJECT,DQS_STAGING...
i am learning SSDT from this link https://msdn.microsoft.com/en-us/library/jj819740(v=sql.120).aspx on point 8 i stuck before that i install DQS successfully and also connected to Data Quality service client now when i check in MS SQL 2014 there is 3 database created DQS_MAIN,DQS_PROJECT,DQS_STAGING_dATA now when i follow point 8 and click on mapping tab this shows an error image any solutions please?
user progam (11 rep)
Aug 3, 2017, 08:47 AM
2 votes
0 answers
31 views
How to retrieve the distinct domains of a compound domain from DQS using SSIS?
MS Data Quality Services (DQS) has a concept of Compound Domains, which lets you turn say "'99 Chevy Maliboo" into {year:1999, brand:Chevrolet, model:Malibu} SSIS has a "DQS Cleansing" component. It only seems to let you retrieve one column back, not the distinct fields of the compound domain. Is th...
MS Data Quality Services (DQS) has a concept of Compound Domains, which lets you turn say "'99 Chevy Maliboo" into {year:1999, brand:Chevrolet, model:Malibu} SSIS has a "DQS Cleansing" component. It only seems to let you retrieve one column back, not the distinct fields of the compound domain. Is there a way to do this? If not, is there a reasonably simple way to do this with a Script Component? Version info: Microsoft Visual Studio 2013 Shell (Integrated) Version 12.0.40629.00 Update 5 Microsoft .NET Framework Version 4.6.00081 Installed Version: IDE Standard Visual C# 2013 06154-004-0430007-02752 Microsoft Visual C# 2013 SQL Server Data Tools 12.0.50730.0 Microsoft SQL Server Data Tools SQL Server Integration Services Microsoft SQL Server Integration Services Designer Version 12.0.2430.0
Neil McGuigan (8653 rep)
May 6, 2016, 12:39 AM
Showing page 1 of 6 total questions