更改

跳到导航 跳到搜索

RWKV模型 (查看源代码)

2025年5月20日 (二) 20:13的版本

添加5,972字节 、 2025年5月20日 (二) 20:13
→‎RWKV 与其他模型对比
第150行： 第150行：  
|}
 
|}
    +
== RWKV 的评测数据 ==
 +
以下是 RWKV 和其他模型的 Uncheatable Eval 评分对比：
 +
 +
=== 14B 参数模型 ===
 +
{| class="wikitable"
 +
!Name
 +
!Params (B)
 +
!Average (lower=better)
 +
!ao3 english
 +
!bbc news
 +
!wikipedia english
 +
!arxiv computer science
 +
!arxiv physics
 +
!github cpp
 +
!github python
 +
|-
 +
|Mistral-Nemo-Base-2407
 +
|12.248
 +
|7.107
 +
|10.07
 +
|8.081
 +
|7.954
 +
|7.419
 +
|7.656
 +
|4.203
 +
|4.368
 +
|-
 +
|RWKV-6-14B-v2.1
 +
|14.069
 +
|7.609
 +
|10.188
 +
|8.518
 +
|8.343
 +
|7.916
 +
|8.04
 +
|4.93
 +
|5.33
 +
|-
 +
|Llama-2-13b-hf
 +
|13.016
 +
|7.676
 +
|10.524
 +
|8.279
 +
|8.187
 +
|8.075
 +
|8.311
 +
|4.929
 +
|5.426
 +
|-
 +
|Qwen1.5-14B
 +
|14.167
 +
|7.697
 +
|10.88
 +
|8.884
 +
|9.102
 +
|7.752
 +
|7.862
 +
|4.665
 +
|4.736
 +
|-
 +
|pythia-12b-v0
 +
|11.846
 +
|8.356
 +
|11.285
 +
|9.19
 +
|9.527
 +
|8.535
 +
|8.398
 +
|5.43
 +
|6.125
 +
|}
 +
 +
=== 7B 参数模型 ===
 +
{| class="wikitable"
 +
!Name
 +
!Params (B)
 +
!Average (lower=better)
 +
!ao3 ​english
 +
!bbc ​news
 +
!wikipedia ​english
 +
!arxiv ​computer ​science
 +
!arxiv ​physics
 +
!github ​cpp
 +
!github ​python
 +
|-
 +
|Meta-Llama-3.1-8B
 +
|8.030
 +
|7.227
 +
|10.529
 +
|8.201
 +
|7.932
 +
|7.549
 +
|7.686
 +
|4.204
 +
|4.487
 +
|-
 +
|Qwen2.5-7B
 +
|7.616
 +
|7.453
 +
|10.931
 +
|8.742
 +
|8.815
 +
|7.683
 +
|7.991
 +
|3.920
 +
|4.091
 +
|-
 +
|Qwen2-7B
 +
|7.616
 +
|7.528
 +
|10.814
 +
|8.582
 +
|8.709
 +
|7.822
 +
|8.249
 +
|4.204
 +
|4.317
 +
|-
 +
|Mistral-7B-v0.1
 +
|7.242
 +
|7.580
 +
|10.546
 +
|8.282
 +
|8.435
 +
|7.852
 +
|8.092
 +
|4.800
 +
|5.051
 +
|-
 +
|RWKV-6-World-7B-v2.1
 +
|7.636
 +
|7.817
 +
|10.410
 +
|8.740
 +
|8.577
 +
|8.107
 +
|8.248
 +
|5.122
 +
|5.516
 +
|-
 +
|Yi-1.5-6B
 +
|6.061
 +
|7.830
 +
|10.926
 +
|8.789
 +
|8.948
 +
|8.098
 +
|8.410
 +
|4.746
 +
|4.890
 +
|-
 +
|OLMo-1.7-7B-hf
 +
|6.888
 +
|7.881
 +
|11.012
 +
|8.647
 +
|8.987
 +
|8.004
 +
|8.200
 +
|4.966
 +
|5.354
 +
|-
 +
|RWKV-5-World-7B-v2
 +
|7.518
 +
|7.905
 +
|10.495
 +
|8.878
 +
|8.694
 +
|8.184
 +
|8.311
 +
|5.187
 +
|5.587
 +
|-
 +
|Qwen1.5-7B
 +
|7.721
 +
|7.916
 +
|11.100
 +
|9.126
 +
|9.357
 +
|7.953
 +
|8.107
 +
|4.845
 +
|4.924
 +
|-
 +
|mpt-7b
 +
|6.649
 +
|7.950
 +
|11.185
 +
|8.682
 +
|8.771
 +
|8.157
 +
|8.438
 +
|4.954
 +
|5.466
 +
|-
 +
|Llama-2-7b-hf
 +
|6.738
 +
|7.967
 +
|10.842
 +
|8.511
 +
|8.521
 +
|8.330
 +
|8.630
 +
|5.235
 +
|5.703
 +
|-
 +
|Zamba-7B-v1
 +
|7.232
 +
|8.089
 +
|10.846
 +
|8.520
 +
|8.639
 +
|8.060
 +
|8.290
 +
|5.938
 +
|6.328
 +
|-
 +
|open_llama_7b_v2
 +
|6.738
 +
|8.103
 +
|11.086
 +
|8.839
 +
|9.053
 +
|8.404
 +
|8.757
 +
|4.887
 +
|5.698
 +
|-
 +
|falcon-7b
 +
|6.922
 +
|8.302
 +
|10.756
 +
|8.691
 +
|9.147
 +
|8.549
 +
|9.060
 +
|5.763
 +
|6.149
 +
|-
 +
|pythia-6.9b-v0
 +
|6.857
 +
|8.545
 +
|11.495
 +
|9.375
 +
|9.757
 +
|8.683
 +
|8.572
 +
|5.611
 +
|6.321
 +
|-
 +
|mamba-7b-rw
 +
|6.947
 +
|9.782
 +
|10.808
 +
|8.548
 +
|8.990
 +
|8.613
 +
|9.142
 +
|11.107
 +
|11.268
 +
|}
 +
 +
=== 3B 参数模型 ===
 +
{| class="wikitable"
 +
!Name
 +
!Params (B)
 +
!Average (lower=better)
 +
!ao3 ​english
 +
!bbc ​news
 +
!wikipedia ​english
 +
!arxiv ​computer ​science
 +
!arxiv ​physics
 +
!github ​cpp
 +
!github ​python
 +
|-
 +
|Llama-3.2-3B
 +
|3.213
 +
|7.573
 +
|10.897
 +
|8.702
 +
|8.283
 +
|7.784
 +
|8.101
 +
|4.590
 +
|4.654
 +
|-
 +
|Qwen2.5-3B
 +
|3.086
 +
|7.663
 +
|11.226
 +
|9.154
 +
|8.959
 +
|7.787
 +
|8.246
 +
|4.119
 +
|4.151
 +
|-
 +
|RWKV-x070-World-2.9B-v3-20250211-ctx4096
 +
|2.948
 +
|7.737
 +
|10.481
 +
|8.919
 +
|8.470
 +
|7.904
 +
|8.337
 +
|4.882
 +
|5.164
 +
|-
 +
|stablelm-3b-4e1t
 +
|2.795
 +
|7.856
 +
|10.887
 +
|8.818
 +
|8.509
 +
|8.153
 +
|8.500
 +
|4.847
 +
|5.281
 +
|-
 +
|recurrentgemma-2b
 +
|2.683
 +
|7.987
 +
|11.302
 +
|8.938
 +
|8.883
 +
|8.237
 +
|8.520
 +
|4.802
 +
|5.224
 +
|-
 +
|RWKV-x060-World-3B-v2.1-20240417-ctx4096
 +
|3.100
 +
|8.078
 +
|10.674
 +
|9.174
 +
|8.816
 +
|8.268
 +
|8.576
 +
|5.385
 +
|5.655
 +
|-
 +
|gemma-2-2b
 +
|2.614
 +
|8.122
 +
|11.352
 +
|8.898
 +
|9.031
 +
|8.385
 +
|8.814
 +
|5.012
 +
|5.364
 +
|-
 +
|mamba2attn-2.7b
 +
|2.698
 +
|8.183
 +
|11.126
 +
|9.279
 +
|9.263
 +
|8.325
 +
|8.287
 +
|5.223
 +
|5.780
 +
|-
 +
|RWKV-5-World-3B-v2-20231113-ctx4096
 +
|3.063
 +
|8.227
 +
|10.831
 +
|9.356
 +
|9.004
 +
|8.415
 +
|8.698
 +
|5.509
 +
|5.776
 +
|-
 +
|mamba2-2.7b
 +
|2.703
 +
|8.292
 +
|11.213
 +
|9.370
 +
|9.384
 +
|8.427
 +
|8.371
 +
|5.343
 +
|5.933
 +
|-
 +
|Zamba2-2.7B
 +
|2.662
 +
|8.317
 +
|10.970
 +
|8.948
 +
|8.743
 +
|8.169
 +
|8.699
 +
|6.393
 +
|6.295
 +
|-
 +
|mamba-2.8b-hf
 +
|2.768
 +
|8.414
 +
|11.311
 +
|9.486
 +
|9.530
 +
|8.566
 +
|8.517
 +
|5.461
 +
|6.027
 +
|-
 +
|RWKV-4-World-3B-v1-20230619-ctx4096
 +
|3.063
 +
|8.525
 +
|10.898
 +
|9.565
 +
|9.301
 +
|8.902
 +
|9.274
 +
|5.671
 +
|6.065
 +
|-
 +
|pythia-2.8b-v0
 +
|2.775
 +
|8.667
 +
|11.664
 +
|9.741
 +
|9.815
 +
|8.724
 +
|8.729
 +
|5.709
 +
|6.288
 +
|-
 +
|granite-3.2-2b-instruct
 +
|2.534
 +
|8.800
 +
|11.961
 +
|10.119
 +
|10.309
 +
|8.997
 +
|9.254
 +
|5.502
 +
|5.460
 +
|-
 +
|granite-3.1-2b-instruct
 +
|2.534
 +
|8.837
 +
|12.004
 +
|10.169
 +
|10.361
 +
|9.043
 +
|9.278
 +
|5.519
 +
|5.484
 +
|}
 +
 +
=== 1.5B 参数模型 ===
 +
{| class="wikitable"
 +
!Name
 +
!Params (B)
 +
!Average (lower=better)
 +
!ao3 ​english
 +
!bbc ​news
 +
!wikipedia ​english
 +
!arxiv ​computer ​science
 +
!arxiv ​physics
 +
!github ​cpp
 +
!github ​python
 +
|-
 +
|Qwen3-1.7B-Base
 +
|1.721
 +
|7.915
 +
|11.644
 +
|9.761
 +
|9.415
 +
|7.816
 +
|8.377
 +
|4.177
 +
|4.214
 +
|-
 +
|rwkv7-g1-1.5b-20250429-ctx4096
 +
|1.528
 +
|7.975
 +
|10.622
 +
|9.269
 +
|8.936
 +
|8.069
 +
|8.649
 +
|4.973
 +
|5.308
 +
|-
 +
|Qwen2.5-1.5B
 +
|1.544
 +
|8.058
 +
|11.756
 +
|9.576
 +
|9.489
 +
|8.118
 +
|8.653
 +
|4.396
 +
|4.421
 +
|-
 +
|RWKV-x070-World-1.5B-v3-20250127-ctx4096
 +
|1.528
 +
|8.160
 +
|10.932
 +
|9.337
 +
|8.972
 +
|8.250
 +
|8.770
 +
|5.288
 +
|5.571
 +
|-
 +
|Llama-3.2-1B
 +
|1.236
 +
|8.225
 +
|11.693
 +
|9.343
 +
|9.066
 +
|8.365
 +
|8.762
 +
|5.160
 +
|5.184
 +
|-
 +
|SmolLM2-1.7B
 +
|1.711
 +
|8.227
 +
|11.204
 +
|9.396
 +
|9.459
 +
|8.382
 +
|9.041
 +
|4.937
 +
|5.172
 +
|-
 +
|Index-1.9B
 +
|2.173
 +
|8.301
 +
|11.490
 +
|9.510
 +
|9.227
 +
|8.344
 +
|8.589
 +
|5.292
 +
|5.652
 +
|-
 +
|stablelm-2-1_6b
 +
|1.645
 +
|8.338
 +
|11.423
 +
|9.243
 +
|9.057
 +
|8.579
 +
|9.076
 +
|5.450
 +
|5.535
 +
|-
 +
|RWKV-x060-World-1B6-v2.1-20240328-ctx4096
 +
|1.600
 +
|8.491
 +
|11.094
 +
|9.569
 +
|9.302
 +
|8.619
 +
|8.997
 +
|5.798
 +
|6.061
 +
|-
 +
|RWKV-5-World-1B5-v2-20231025-ctx4096
 +
|1.578
 +
|8.640
 +
|11.247
 +
|9.746
 +
|9.499
 +
|8.765
 +
|9.106
 +
|5.918
 +
|6.196
 +
|-
 +
|mamba2-1.3b
 +
|1.344
 +
|8.676
 +
|11.625
 +
|9.743
 +
|9.863
 +
|8.741
 +
|8.742
 +
|5.706
 +
|6.315
 +
|-
 +
|MobileLLM-1.5B
 +
|1.562
 +
|8.733
 +
|11.588
 +
|9.145
 +
|9.215
 +
|8.816
 +
|9.292
 +
|6.293
 +
|6.785
 +
|-
 +
|mamba-1.4b-hf
 +
|1.372
 +
|8.783
 +
|11.703
 +
|9.828
 +
|9.973
 +
|8.876
 +
|8.863
 +
|5.808
 +
|6.429
 +
|-
 +
|Zamba2-1.2B
 +
|1.215
 +
|8.830
 +
|11.392
 +
|9.382
 +
|9.263
 +
|8.571
 +
|9.209
 +
|7.082
 +
|6.914
 +
|-
 +
|SmolLM-1.7B
 +
|1.711
 +
|8.877
 +
|12.684
 +
|9.847
 +
|9.890
 +
|8.384
 +
|9.024
 +
|6.552
 +
|5.757
 +
|-
 +
|gemma-3-1b-pt
 +
|1.000
 +
|8.903
 +
|12.188
 +
|9.364
 +
|9.597
 +
|8.874
 +
|9.559
 +
|6.591
 +
|6.150
 +
|-
 +
|MobileLLM-1B
 +
|1.005
 +
|8.972
 +
|11.863
 +
|9.349
 +
|9.432
 +
|9.028
 +
|9.566
 +
|6.531
 +
|7.033
 +
|-
 +
|RWKV-4-World-1.5B-v1-fixed-20230612-ctx4096
 +
|1.578
 +
|8.999
 +
|11.334
 +
|10.000
 +
|9.821
 +
|9.344
 +
|9.797
 +
|6.158
 +
|6.539
 +
|-
 +
|Hymba-1.5B-Base
 +
|1.523
 +
|9.125
 +
|11.797
 +
|9.446
 +
|9.502
 +
|8.497
 +
|9.375
 +
|7.714
 +
|7.542
 +
|-
 +
|pythia-1.4b-v0
 +
|1.415
 +
|9.154
 +
|12.192
 +
|10.200
 +
|10.429
 +
|9.122
 +
|9.198
 +
|6.149
 +
|6.788
 +
|-
 +
|Falcon3-1B-Base
 +
|1.669
 +
|9.445
 +
|13.039
 +
|10.448
 +
|10.750
 +
|8.604
 +
|9.200
 +
|7.155
 +
|6.919
 +
|}
 
----
 
----
 
== RWKV 与其他模型对比 ==
 
== RWKV 与其他模型对比 ==
RWKV
201

个编辑

取自“https://wiki.swarma.org/index.php/特殊:移动版差异/44051”

导航菜单