Skip to content

[Bug] sampling rate of dynamic configuration using DCS does not work #13836

Closed

Description

@ranxuxin001

Search before asking

  • I had searched in the issues and found no similar issues.

Apache SkyWalking Component

OAP server (apache/skywalking)

What happened

Hello,

I've written the call() function of my grpc server to configure sampling rate. Oap log shows it calls call() function and parse the sampling rate correctly. But skywalking ui shows the same trace frequecy.

What you expected to happen

the sampling rate dynamic configuration works via grpc.

How to reproduce

My yaml configuration shows below:

const agentConfigurationYAML = `default:
  rate: 10000
  duration: -1
services:
  - name: qewqw-test::myapmmysqlservice
    rate: 100
    duration: -1`

func GetFromMysql() string {
	//query mysql to get configuration.
	return agentConfigurationYAML
}

func (s *configServer) Call(ctx context.Context, req *pb.ConfigurationRequest) (*pb.ConfigurationResponse, error) {
	configValue := GetFromMysql()
	log.Println("config value: ", configValue)
	cfgTb := []*pb.Config{
		{
			Name: "agent-analyzer.default.traceSamplingPolicy",
			Value: configValue,
		},

.....

        response := &pb.ConfigurationResponse{
		ConfigTable: cfgTb,
		Uuid:        s.currentUUID,
	}

	fmt.Printf("single. will send response to oap. %v", response.ConfigTable)
	return response, nil
}

Anything else

No response

Are you willing to submit a pull request to fix on your own?

  • Yes I am willing to submit a pull request on my own!

Code of Conduct

Activity

  1. added
    bugSomething isn't working and you are sure it's a bug!
    on Apr 21, 2026
  2. wu-sheng commented on Apr 21, 2026

    @wu-sheng
    Member

    Hi @ranxuxin001, before we dig into this as a bug, two questions — either one can fully explain what you're seeing:

    1. Does s.currentUUID change every time the underlying MySQL config changes?

    The DCS gRPC protocol is version-based, not push-based. See GRPCConfigWatcherRegister#readConfig:

    if (Objects.equals(uuid, responseUuid)) {
        // If UUID matched, the config table is expected as empty.
        return Optional.empty();
    }

    The OAP sends the last UUID it saw; if your server returns the same UUID, the OAP treats the poll as "no change" and never re-notifies the watcher — even if ConfigTable carries a new value. Your log showing call() being hit and the value being parsed is consistent with the first poll succeeding and every subsequent poll being ignored. Your server must bump the UUID (or hash the payload, or use the row's updated_at) whenever the value bytes change.

    2. Is qewqw-test::myapmmysqlservice actually an agent-reported service name, or is it a virtual database node in the topology?

    TraceSegmentSampler#shouldSample matches on segmentObject.getService() — the name the agent sets for itself, not any synthesized entity. The :: pattern usually shows up on conjured nodes like virtual databases / MQ / caches that SkyWalking derives from span relations; those are never carried as a segment's service field, so a services[].name rule targeting them will never match. If that's the case here, your rate: 100 override is silently ignored and only default.rate: 10000 (= 100%) applies — which is exactly "UI shows the same trace frequency."

    Could you confirm both? If (1) is the cause, the fix is on your gRPC server side. If (2) is the cause, you'll want to put the agent's own service name in services[].name instead.

  3. locked and limited conversation to collaborators on Apr 21, 2026
  4. converted this issue into a discussion #13841 on Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working and you are sure it's a bug!

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions