• State Resolved
  • Locked Locked
  • Replies 4 replies
  • Answers 1 answer
  • Subscribers 101 subscribers
  • Views 382 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA2PXEVM: What do the sparsity parameters mean in solver.prototxt when do sparse training

lluo
Intellectual 340 points
Part Number: TDA2PXEVM

Hi, 

I am doing sparse training after stage L1 on caffe-jacinto.

I checked the solver_train.prototxt in trained model. There are some parameters for sparse training but I don't get the meaning of them. Those parameters are showed below:

display_sparsity: 2000
sparse_mode: SPARSE_UPDATE
sparsity_target: 0.7
sparsity_step_factor: 0.05
sparsity_step_iter: 2000
sparsity_start_iter: 0
sparsity_start_factor: 0.25
sparsity_threshold_maxratio: 0.2
sparsity_itr_increment_bfr_applying: true
sparsity_threshold_value_max: 0.2

I can understand display_sparsity, sparsity_target, but I'm confused about the others. 

Can you describe how to use sparse training parameters in solver.prototxt ?

Is there any description or tips on parameters of sparse training ?

 

  • 0 in reply to Yordan Kamenov
    Intellectual 340 points
    Hi, Yordan
    I checked the paper, but it doesn't mention sparse training parameters to be set in solver.prototxt.
    Can you explain the meaning of those parameters in solver.prototxt on sparse training stage?

    display_sparsity: 2000
    sparse_mode: SPARSE_UPDATE
    sparsity_target: 0.7
    sparsity_step_factor: 0.05
    sparsity_step_iter: 2000
    sparsity_start_iter: 0
    sparsity_start_factor: 0.25
    sparsity_threshold_maxratio: 0.2
    sparsity_itr_increment_bfr_applying: true
    sparsity_threshold_value_max: 0.2

    Regards,
    Geroge
  • 0 in reply to lluo
    TI__Genius 9926 points

    Hi, you can find brief descripts on caffe.proto, where these parameters are defined.

    https://github.com/tidsp/caffe-jacinto/blob/caffe-0.17/src/caffe/proto/caffe.proto#L521

    // Sparsity params
    optional int32 display_sparsity = 152 [default = 0];
    optional SparseMode sparse_mode = 153 [default = SPARSE_NONE];
    optional float sparsity_target = 154 [default = 0.0]; //desired sparsity as a fraction
    optional float sparsity_step_factor = 155 [default = 0.01]; //sprsity step factor. 0.01 is 1%
    optional int32 sparsity_step_iter = 156 [default = 1000]; //sprsity step increment iterations
    optional int32 sparsity_start_iter = 157 [default = 0]; //sprsity start iteration
    optional float sparsity_start_factor = 158 [default = 0.0]; //sprsity start factor
    optional float sparsity_threshold_maxratio = 159 [default = 0.2]; //default ratio of the max threshold allowed to the max weight
    optional bool sparsity_itr_increment_bfr_applying = 160 [default = true]; //true:itr increment before applying sparsity(old behaviour), false: after applying sparsity
    optional float sparsity_threshold_value_max = 161 [default = 0.2]; //threshold_value_max

    I hope this helps. Let me know if you have a doubt on any specific parameter.

    Best regards,

  • 0 in reply to Manu Mathew
    Intellectual 340 points
    Oh, this really helps！
    Thanks!