Migrate VectorStoreRAG and Concepts samples to CommunityToolkit.VectorData packages#14170
Migrate VectorStoreRAG and Concepts samples to CommunityToolkit.VectorData packages#14170adamsitnik with Copilot wants to merge 9 commits into
Conversation
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot please address my feedback
…press/Snappier pins
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot fix following build failure:
/home/runner/work/semantic-kernel/semantic-kernel/dotnet/samples/GettingStartedWithVectorStores/GettingStartedWithVectorStores.csproj : error NU1010: The following PackageReference items do not define a corresponding PackageVersion item: Microsoft.SemanticKernel.Connectors.AzureAISearch, Microsoft.SemanticKernel.Connectors.InMemory, Microsoft.SemanticKernel.Connectors.Redis. Projects using Central Package Management must declare PackageReference and PackageVersion items with matching names. For more information, visit https://aka.ms/nuget/cpm/gettingstarted [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/SK-dotnet.slnx]
/home/runner/work/semantic-kernel/semantic-kernel/dotnet/samples/GettingStartedWithTextSearch/GettingStartedWithTextSearch.csproj : error NU1010: The following PackageReference items do not define a corresponding PackageVersion item: Microsoft.SemanticKernel.Connectors.InMemory. Projects using Central Package Management must declare PackageReference and PackageVersion items with matching names. For more information, visit https://aka.ms/nuget/cpm/gettingstarted [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/SK-dotnet.slnx]
/home/runner/work/semantic-kernel/semantic-kernel/dotnet/samples/Demos/OnnxSimpleRAG/OnnxSimpleRAG.csproj : error NU1010: The following PackageReference items do not define a corresponding PackageVersion item: Microsoft.SemanticKernel.Connectors.InMemory. Projects using Central Package Management must declare PackageReference and PackageVersion items with matching names. For more information, visit https://aka.ms/nuget/cpm/gettingstarted [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/SK-dotnet.slnx]
/home/runner/work/semantic-kernel/semantic-kernel/dotnet/samples/Demos/ModelContextProtocolClientServer/MCPServer/MCPServer.csproj : error NU1010: The following PackageReference items do not define a corresponding PackageVersion item: Microsoft.SemanticKernel.Connectors.InMemory. Projects using Central Package Management must declare PackageReference and PackageVersion items with matching names. For more information, visit https://aka.ms/nuget/cpm/gettingstarted [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/SK-dotnet.slnx]
/home/runner/work/semantic-kernel/semantic-kernel/dotnet/samples/Demos/AgentFrameworkWithAspire/ChatWithAgent.ApiService/ChatWithAgent.ApiService.csproj : error NU1010: The following PackageReference items do not define a corresponding PackageVersion item: Microsoft.SemanticKernel.Connectors.AzureAISearch. Projects using Central Package Management must declare PackageReference and PackageVersion items with matching names. For more information, visit https://aka.ms/nuget/cpm/gettingstarted [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/SK-dotnet.slnx]
/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.UnitTests/SemanticKernel.UnitTests.csproj : error NU1010: The following PackageReference items do not define a corresponding PackageVersion item: Microsoft.SemanticKernel.Connectors.InMemory. Projects using Central Package Management must declare PackageReference and PackageVersion items with matching names. For more information, visit https://aka.ms/nuget/cpm/gettingstarted [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/SK-dotnet.slnx]
make sure that none of the projects in this repo keeps using Microsoft.SemanticKernel.Connectors.* packages, just migrate all of them to CommunityToolkit.VectorData.*
Fixed in the latest commit. All 6 failing projects have been migrated:
|
…ry package reference
Fixed in the latest commit:
|
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot fix following test failure:
Failed SemanticKernel.UnitTests.Data.TextSearchServiceCollectionExtensionsTests.AddVectorStoreTextSearchWithNoMappers [8 ms]
[xUnit.net 00:00:02.18] System.TypeLoadException : GenericArguments[0], 'TRecord', on 'Microsoft.Extensions.VectorData.IVectorSearchable`1[TRecord]' violates the constraint of type parameter 'TRecord'.
[xUnit.net 00:00:02.18] Stack Trace:
[xUnit.net 00:00:02.18] /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.Core/Data/TextSearch/TextSearchServiceCollectionExtensions.cs(38,0): at Microsoft.SemanticKernel.TextSearchServiceCollectionExtensions.<>c__DisplayClass0_0`1.<AddVectorStoreTextSearch>b__0(IServiceProvider sp, Object obj)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceDescriptor.<>c__DisplayClass5_0.<.ctor>b__0(IServiceProvider sp)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
[xUnit.net 00:00:02.18] at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
[xUnit.net 00:00:02.18] /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.UnitTests/Data/TextSearchServiceCollectionExtensionsTests.cs(56,0): at SemanticKernel.UnitTests.Data.TextSearchServiceCollectionExtensionsTests.AddVectorStoreTextSearchWithNoMappers()
[xUnit.net 00:00:02.18] at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
[xUnit.net 00:00:02.18] at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Fixed in the latest commit. Root cause: Changes made:
All 1609 unit tests pass. |
There was a problem hiding this comment.
@copilot why do we need this file? It seems to be redundant.
There was a problem hiding this comment.
The file is required. Here's the chain:
CommunityToolkit.VectorData.InMemory1.0.0-preview.3 depends onMicrosoft.Extensions.VectorData.Abstractions10.5.2, which added awhere TRecord : classconstraint toIVectorSearchable<TRecord>.VectorStoreTextSearch<TRecord>holds aprivate readonly IVectorSearchable<TRecord>?field and accepts it as constructor parameters — the CLR requires the constraint to be propagated, otherwise it throwsTypeLoadExceptionat JIT time (which was the actual unit-test failure you reported earlier).- Adding
where TRecord : classtoVectorStoreTextSearch<TRecord>,AddVectorStoreTextSearch, andAddVectorStoreTextSearchis a binary-breaking API change relative to the 1.77.0 baseline (which compiled against 10.1.0 where the constraint didn't exist). - Package validation (enabled for Release/Publish builds via
<EnablePackageValidation>true</EnablePackageValidation>) therefore raises 9CP0021errors —CompatibilitySuppressions.xmlsuppresses them.
I verified this by removing the file and running dotnet build --configuration Release on SemanticKernel.Core.csproj — it fails with exactly those 9 CP0021 errors. The file is not redundant.
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot fix following build failures:
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(7,23): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'IVectorSearchable<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net10.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(9,68): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net10.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(11,80): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net10.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(16,58): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net10.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(26,58): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net10.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(7,23): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'IVectorSearchable<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net9.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(9,68): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net9.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(11,80): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net9.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(16,58): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net9.0]
Error: /home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/UnitTests/Search/MockVectorizableTextSearch.cs(26,58): error CS0452: The type 'TRecord' must be a reference type in order to use it as parameter 'TRecord' in the generic type or method 'VectorSearchResult<TRecord>' [/home/runner/work/semantic-kernel/semantic-kernel/dotnet/src/SemanticKernel.AotTests/SemanticKernel.AotTests.csproj::TargetFramework=net9.0]
@copilot make sure you build dotnet/SK-dotnet.slnx succesfully before pushing any changes.
Fixed in the latest commit. Added |
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot fix the last build failure:
/app/dotnet/src/InternalUtilities/src/System/PathUtilities.cs(6,1): warning IDE0005: Using directive is unnecessary. [/app/dotnet/src/VectorData/InMemory/InMemory.csproj]
/app/dotnet/src/InternalUtilities/src/System/PathUtilities.cs(6,1): warning IDE0005: Using directive is unnecessary. [/app/dotnet/src/VectorData/Weaviate/Weaviate.csproj]
Fixed in the latest commit. |
Motivation and Context
Microsoft.SemanticKernel.Connectors.*vector store packages are moving toCommunityToolkit.VectorData.*. This updates theVectorStoreRAGandConceptssample projects to reference the new package IDs and namespaces.Description
Package reference updates (
Directory.Packages.props,VectorStoreRAG.csproj,Concepts.csproj):Microsoft.SemanticKernel.Connectors.AzureAISearchCommunityToolkit.VectorData.AzureAISearchMicrosoft.SemanticKernel.Connectors.CosmosMongoDBCommunityToolkit.VectorData.CosmosMongoDBMicrosoft.SemanticKernel.Connectors.CosmosNoSqlCommunityToolkit.VectorData.CosmosNoSqlMicrosoft.SemanticKernel.Connectors.InMemoryCommunityToolkit.VectorData.InMemoryMicrosoft.SemanticKernel.Connectors.PgVectorCommunityToolkit.VectorData.PgVectorMicrosoft.SemanticKernel.Connectors.PineconeCommunityToolkit.VectorData.PineconeMicrosoft.SemanticKernel.Connectors.QdrantCommunityToolkit.VectorData.QdrantMicrosoft.SemanticKernel.Connectors.RedisCommunityToolkit.VectorData.RedisMicrosoft.SemanticKernel.Connectors.WeaviateCommunityToolkit.VectorData.WeaviateNamespace updates (24
.csfiles inConcepts/):DI extension methods (
AddInMemoryVectorStore,AddQdrantCollection, etc.) moved toMicrosoft.Extensions.DependencyInjectionin the CT packages — all affected files already had thatusing, so no additional changes needed there.API compatibility fixes:
[VectorStoreVector(Dimensions: N)]→[VectorStoreVector(N)]in two files — the newMicrosoft.Extensions.VectorData.Abstractionsconstructor uses a positional parameter nameddimensions(lowercase), so the old named-argument form no longer compiles.SharpCompresspin bumped0.48.0→0.48.1inDirectory.Packages.props—CommunityToolkit.VectorData.CosmosMongoDBpullsMongoDB.Driver 3.10.0which requires>= 0.48.1.<AzureCosmosDisableNewtonsoftJsonCheck>true</AzureCosmosDisableNewtonsoftJsonCheck>to both sample csproj files —CommunityToolkit.VectorData.CosmosNoSqlpullsMicrosoft.Azure.Cosmos 3.61.0which added a mandatory Newtonsoft.Json explicit-reference check not present in the prior version.Contribution Checklist