myrag/.model/nomic-embed-text-v1.5/README.md

library_name, pipeline_tag, tags, model-index, license, language

library_name	pipeline_tag	tags	model-index	license	language
sentence-transformers	sentence-similarity	feature-extraction sentence-similarity mteb transformers transformers.js	name results epoch_0_model task dataset metrics type Classification type name config split revision mteb/amazon_counterfactual MTEB AmazonCounterfactualClassification (en) en test e8379541af4e31359cca9fbcf4b00f2671dba205 type value accuracy 75.20895522388058 type value ap 38.57605549557802 type value f1 69.35586565857854 task dataset metrics type Classification type name config split revision mteb/amazon_polarity MTEB AmazonPolarityClassification default test e2d317d38cd51312af73b3d32a06d1a08b442046 type value accuracy 91.8144 type value ap 88.65222882032363 type value f1 91.80426301643274 task dataset metrics type Classification type name config split revision mteb/amazon_reviews_multi MTEB AmazonReviewsClassification (en) en test 1399c76144fd37290681b995c656ef9b2e06e26d type value accuracy 47.162000000000006 type value f1 46.59329642263158 task dataset metrics type Retrieval type name config split revision arguana MTEB ArguAna default test None type value map_at_1 24.253 type value map_at_10 38.962 type value map_at_100 40.081 type value map_at_1000 40.089000000000006 type value map_at_3 33.499 type value map_at_5 36.351 type value mrr_at_1 24.609 type value mrr_at_10 39.099000000000004 type value mrr_at_100 40.211000000000006 type value mrr_at_1000 40.219 type value mrr_at_3 33.677 type value mrr_at_5 36.469 type value ndcg_at_1 24.253 type value ndcg_at_10 48.010999999999996 type value ndcg_at_100 52.756 type value ndcg_at_1000 52.964999999999996 type value ndcg_at_3 36.564 type value ndcg_at_5 41.711999999999996 type value precision_at_1 24.253 type value precision_at_10 7.738 type value precision_at_100 0.98 type value precision_at_1000 0.1 type value precision_at_3 15.149000000000001 type value precision_at_5 11.593 type value recall_at_1 24.253 type value recall_at_10 77.383 type value recall_at_100 98.009 type value recall_at_1000 99.644 type value recall_at_3 45.448 type value recall_at_5 57.965999999999994 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-p2p MTEB ArxivClusteringP2P default test a122ad7f3f0291bf49cc6f4d32aa80929df69d5d type value v_measure 45.69069567851087 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-s2s MTEB ArxivClusteringS2S default test f910caf1a6075f7329cdf8c1a6135696f37dbd53 type value v_measure 36.35185490976283 task dataset metrics type Reranking type name config split revision mteb/askubuntudupquestions-reranking MTEB AskUbuntuDupQuestions default test 2000358ca161889fa9c082cb41daa8dcfb161a54 type value map 61.71274951450321 type value mrr 76.06032625423207 task dataset metrics type STS type name config split revision mteb/biosses-sts MTEB BIOSSES default test d3fb88f8f02e40887cd149695127462bbcf29b4a type value cos_sim_pearson 86.73980520022269 type value cos_sim_spearman 84.24649792685918 type value euclidean_pearson 85.85197641158186 type value euclidean_spearman 84.24649792685918 type value manhattan_pearson 86.26809552711346 type value manhattan_spearman 84.56397504030865 task dataset metrics type Classification type name config split revision mteb/banking77 MTEB Banking77Classification default test 0fd18e25b25c072e09e0d92ab615fda904d66300 type value accuracy 84.25324675324674 type value f1 84.17872280892557 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-p2p MTEB BiorxivClusteringP2P default test 65b79d1d13f80053f67aca9498d9402c2d9f1f40 type value v_measure 38.770253446400886 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-s2s MTEB BiorxivClusteringS2S default test 258694dd0231531bc1fd9de6ceb52a0853c6d908 type value v_measure 32.94307095497281 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackAndroidRetrieval default test None type value map_at_1 32.164 type value map_at_10 42.641 type value map_at_100 43.947 type value map_at_1000 44.074999999999996 type value map_at_3 39.592 type value map_at_5 41.204 type value mrr_at_1 39.628 type value mrr_at_10 48.625 type value mrr_at_100 49.368 type value mrr_at_1000 49.413000000000004 type value mrr_at_3 46.400000000000006 type value mrr_at_5 47.68 type value ndcg_at_1 39.628 type value ndcg_at_10 48.564 type value ndcg_at_100 53.507000000000005 type value ndcg_at_1000 55.635999999999996 type value ndcg_at_3 44.471 type value ndcg_at_5 46.137 type value precision_at_1 39.628 type value precision_at_10 8.856 type value precision_at_100 1.429 type value precision_at_1000 0.191 type value precision_at_3 21.268 type value precision_at_5 14.649000000000001 type value recall_at_1 32.164 type value recall_at_10 59.609 type value recall_at_100 80.521 type value recall_at_1000 94.245 type value recall_at_3 46.521 type value recall_at_5 52.083999999999996 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackEnglishRetrieval default test None type value map_at_1 31.526 type value map_at_10 41.581 type value map_at_100 42.815999999999995 type value map_at_1000 42.936 type value map_at_3 38.605000000000004 type value map_at_5 40.351 type value mrr_at_1 39.489999999999995 type value mrr_at_10 47.829 type value mrr_at_100 48.512 type value mrr_at_1000 48.552 type value mrr_at_3 45.754 type value mrr_at_5 46.986 type value ndcg_at_1 39.489999999999995 type value ndcg_at_10 47.269 type value ndcg_at_100 51.564 type value ndcg_at_1000 53.53099999999999 type value ndcg_at_3 43.301 type value ndcg_at_5 45.239000000000004 type value precision_at_1 39.489999999999995 type value precision_at_10 8.93 type value precision_at_100 1.415 type value precision_at_1000 0.188 type value precision_at_3 20.892 type value precision_at_5 14.865999999999998 type value recall_at_1 31.526 type value recall_at_10 56.76 type value recall_at_100 75.029 type value recall_at_1000 87.491 type value recall_at_3 44.786 type value recall_at_5 50.254 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGamingRetrieval default test None type value map_at_1 40.987 type value map_at_10 52.827 type value map_at_100 53.751000000000005 type value map_at_1000 53.81 type value map_at_3 49.844 type value map_at_5 51.473 type value mrr_at_1 46.833999999999996 type value mrr_at_10 56.389 type value mrr_at_100 57.003 type value mrr_at_1000 57.034 type value mrr_at_3 54.17999999999999 type value mrr_at_5 55.486999999999995 type value ndcg_at_1 46.833999999999996 type value ndcg_at_10 58.372 type value ndcg_at_100 62.068 type value ndcg_at_1000 63.288 type value ndcg_at_3 53.400000000000006 type value ndcg_at_5 55.766000000000005 type value precision_at_1 46.833999999999996 type value precision_at_10 9.191 type value precision_at_100 1.192 type value precision_at_1000 0.134 type value precision_at_3 23.448 type value precision_at_5 15.862000000000002 type value recall_at_1 40.987 type value recall_at_10 71.146 type value recall_at_100 87.035 type value recall_at_1000 95.633 type value recall_at_3 58.025999999999996 type value recall_at_5 63.815999999999995 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGisRetrieval default test None type value map_at_1 24.587 type value map_at_10 33.114 type value map_at_100 34.043 type value map_at_1000 34.123999999999995 type value map_at_3 30.45 type value map_at_5 31.813999999999997 type value mrr_at_1 26.554 type value mrr_at_10 35.148 type value mrr_at_100 35.926 type value mrr_at_1000 35.991 type value mrr_at_3 32.599000000000004 type value mrr_at_5 33.893 type value ndcg_at_1 26.554 type value ndcg_at_10 38.132 type value ndcg_at_100 42.78 type value ndcg_at_1000 44.919 type value ndcg_at_3 32.833 type value ndcg_at_5 35.168 type value precision_at_1 26.554 type value precision_at_10 5.921 type value precision_at_100 0.8659999999999999 type value precision_at_1000 0.109 type value precision_at_3 13.861 type value precision_at_5 9.605 type value recall_at_1 24.587 type value recall_at_10 51.690000000000005 type value recall_at_100 73.428 type value recall_at_1000 89.551 type value recall_at_3 37.336999999999996 type value recall_at_5 43.047000000000004 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackMathematicaRetrieval default test None type value map_at_1 16.715 type value map_at_10 24.251 type value map_at_100 25.326999999999998 type value map_at_1000 25.455 type value map_at_3 21.912000000000003 type value map_at_5 23.257 type value mrr_at_1 20.274 type value mrr_at_10 28.552 type value mrr_at_100 29.42 type value mrr_at_1000 29.497 type value mrr_at_3 26.14 type value mrr_at_5 27.502 type value ndcg_at_1 20.274 type value ndcg_at_10 29.088 type value ndcg_at_100 34.293 type value ndcg_at_1000 37.271 type value ndcg_at_3 24.708 type value ndcg_at_5 26.809 type value precision_at_1 20.274 type value precision_at_10 5.361 type value precision_at_100 0.915 type value precision_at_1000 0.13 type value precision_at_3 11.733 type value precision_at_5 8.556999999999999 type value recall_at_1 16.715 type value recall_at_10 39.587 type value recall_at_100 62.336000000000006 type value recall_at_1000 83.453 type value recall_at_3 27.839999999999996 type value recall_at_5 32.952999999999996 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackPhysicsRetrieval default test None type value map_at_1 28.793000000000003 type value map_at_10 38.582 type value map_at_100 39.881 type value map_at_1000 39.987 type value map_at_3 35.851 type value map_at_5 37.289 type value mrr_at_1 34.455999999999996 type value mrr_at_10 43.909 type value mrr_at_100 44.74 type value mrr_at_1000 44.786 type value mrr_at_3 41.659 type value mrr_at_5 43.010999999999996 type value ndcg_at_1 34.455999999999996 type value ndcg_at_10 44.266 type value ndcg_at_100 49.639 type value ndcg_at_1000 51.644 type value ndcg_at_3 39.865 type value ndcg_at_5 41.887 type value precision_at_1 34.455999999999996 type value precision_at_10 7.843999999999999 type value precision_at_100 1.243 type value precision_at_1000 0.158 type value precision_at_3 18.831999999999997 type value precision_at_5 13.147 type value recall_at_1 28.793000000000003 type value recall_at_10 55.68300000000001 type value recall_at_100 77.99000000000001 type value recall_at_1000 91.183 type value recall_at_3 43.293 type value recall_at_5 48.618 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackProgrammersRetrieval default test None type value map_at_1 25.907000000000004 type value map_at_10 35.519 type value map_at_100 36.806 type value map_at_1000 36.912 type value map_at_3 32.748 type value map_at_5 34.232 type value mrr_at_1 31.621 type value mrr_at_10 40.687 type value mrr_at_100 41.583 type value mrr_at_1000 41.638999999999996 type value mrr_at_3 38.527 type value mrr_at_5 39.612 type value ndcg_at_1 31.621 type value ndcg_at_10 41.003 type value ndcg_at_100 46.617999999999995 type value ndcg_at_1000 48.82 type value ndcg_at_3 36.542 type value ndcg_at_5 38.368 type value precision_at_1 31.621 type value precision_at_10 7.396999999999999 type value precision_at_100 1.191 type value precision_at_1000 0.153 type value precision_at_3 17.39 type value precision_at_5 12.1 type value recall_at_1 25.907000000000004 type value recall_at_10 52.115 type value recall_at_100 76.238 type value recall_at_1000 91.218 type value recall_at_3 39.417 type value recall_at_5 44.435 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackRetrieval default test None type value map_at_1 25.732166666666668 type value map_at_10 34.51616666666667 type value map_at_100 35.67241666666666 type value map_at_1000 35.78675 type value map_at_3 31.953416666666662 type value map_at_5 33.333 type value mrr_at_1 30.300166666666673 type value mrr_at_10 38.6255 type value mrr_at_100 39.46183333333334 type value mrr_at_1000 39.519999999999996 type value mrr_at_3 36.41299999999999 type value mrr_at_5 37.6365 type value ndcg_at_1 30.300166666666673 type value ndcg_at_10 39.61466666666667 type value ndcg_at_100 44.60808333333334 type value ndcg_at_1000 46.91708333333334 type value ndcg_at_3 35.26558333333333 type value ndcg_at_5 37.220000000000006 type value precision_at_1 30.300166666666673 type value precision_at_10 6.837416666666667 type value precision_at_100 1.10425 type value precision_at_1000 0.14875 type value precision_at_3 16.13716666666667 type value precision_at_5 11.2815 type value recall_at_1 25.732166666666668 type value recall_at_10 50.578916666666665 type value recall_at_100 72.42183333333334 type value recall_at_1000 88.48766666666667 type value recall_at_3 38.41325 type value recall_at_5 43.515750000000004 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackStatsRetrieval default test None type value map_at_1 23.951 type value map_at_10 30.974 type value map_at_100 31.804 type value map_at_1000 31.900000000000002 type value map_at_3 28.762 type value map_at_5 29.94 type value mrr_at_1 26.534000000000002 type value mrr_at_10 33.553 type value mrr_at_100 34.297 type value mrr_at_1000 34.36 type value mrr_at_3 31.391000000000002 type value mrr_at_5 32.525999999999996 type value ndcg_at_1 26.534000000000002 type value ndcg_at_10 35.112 type value ndcg_at_100 39.28 type value ndcg_at_1000 41.723 type value ndcg_at_3 30.902 type value ndcg_at_5 32.759 type value precision_at_1 26.534000000000002 type value precision_at_10 5.445 type value precision_at_100 0.819 type value precision_at_1000 0.11 type value precision_at_3 12.986 type value precision_at_5 9.049 type value recall_at_1 23.951 type value recall_at_10 45.24 type value recall_at_100 64.12299999999999 type value recall_at_1000 82.28999999999999 type value recall_at_3 33.806000000000004 type value recall_at_5 38.277 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackTexRetrieval default test None type value map_at_1 16.829 type value map_at_10 23.684 type value map_at_100 24.683 type value map_at_1000 24.81 type value map_at_3 21.554000000000002 type value map_at_5 22.768 type value mrr_at_1 20.096 type value mrr_at_10 27.230999999999998 type value mrr_at_100 28.083999999999996 type value mrr_at_1000 28.166000000000004 type value mrr_at_3 25.212 type value mrr_at_5 26.32 type value ndcg_at_1 20.096 type value ndcg_at_10 27.989000000000004 type value ndcg_at_100 32.847 type value ndcg_at_1000 35.896 type value ndcg_at_3 24.116 type value ndcg_at_5 25.964 type value precision_at_1 20.096 type value precision_at_10 5 type value precision_at_100 0.8750000000000001 type value precision_at_1000 0.131 type value precision_at_3 11.207 type value precision_at_5 8.08 type value recall_at_1 16.829 type value recall_at_10 37.407000000000004 type value recall_at_100 59.101000000000006 type value recall_at_1000 81.024 type value recall_at_3 26.739 type value recall_at_5 31.524 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackUnixRetrieval default test None type value map_at_1 24.138 type value map_at_10 32.275999999999996 type value map_at_100 33.416000000000004 type value map_at_1000 33.527 type value map_at_3 29.854000000000003 type value map_at_5 31.096 type value mrr_at_1 28.450999999999997 type value mrr_at_10 36.214 type value mrr_at_100 37.134 type value mrr_at_1000 37.198 type value mrr_at_3 34.001999999999995 type value mrr_at_5 35.187000000000005 type value ndcg_at_1 28.450999999999997 type value ndcg_at_10 37.166 type value ndcg_at_100 42.454 type value ndcg_at_1000 44.976 type value ndcg_at_3 32.796 type value ndcg_at_5 34.631 type value precision_at_1 28.450999999999997 type value precision_at_10 6.241 type value precision_at_100 0.9950000000000001 type value precision_at_1000 0.133 type value precision_at_3 14.801 type value precision_at_5 10.280000000000001 type value recall_at_1 24.138 type value recall_at_10 48.111 type value recall_at_100 71.245 type value recall_at_1000 88.986 type value recall_at_3 36.119 type value recall_at_5 40.846 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWebmastersRetrieval default test None type value map_at_1 23.244 type value map_at_10 31.227 type value map_at_100 33.007 type value map_at_1000 33.223 type value map_at_3 28.924 type value map_at_5 30.017 type value mrr_at_1 27.668 type value mrr_at_10 35.524 type value mrr_at_100 36.699 type value mrr_at_1000 36.759 type value mrr_at_3 33.366 type value mrr_at_5 34.552 type value ndcg_at_1 27.668 type value ndcg_at_10 36.381 type value ndcg_at_100 43.062 type value ndcg_at_1000 45.656 type value ndcg_at_3 32.501999999999995 type value ndcg_at_5 34.105999999999995 type value precision_at_1 27.668 type value precision_at_10 6.798 type value precision_at_100 1.492 type value precision_at_1000 0.234 type value precision_at_3 15.152 type value precision_at_5 10.791 type value recall_at_1 23.244 type value recall_at_10 45.979 type value recall_at_100 74.822 type value recall_at_1000 91.078 type value recall_at_3 34.925 type value recall_at_5 39.126 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWordpressRetrieval default test None type value map_at_1 19.945 type value map_at_10 27.517999999999997 type value map_at_100 28.588 type value map_at_1000 28.682000000000002 type value map_at_3 25.345000000000002 type value map_at_5 26.555 type value mrr_at_1 21.996 type value mrr_at_10 29.845 type value mrr_at_100 30.775999999999996 type value mrr_at_1000 30.845 type value mrr_at_3 27.726 type value mrr_at_5 28.882 type value ndcg_at_1 21.996 type value ndcg_at_10 32.034 type value ndcg_at_100 37.185 type value ndcg_at_1000 39.645 type value ndcg_at_3 27.750999999999998 type value ndcg_at_5 29.805999999999997 type value precision_at_1 21.996 type value precision_at_10 5.065 type value precision_at_100 0.819 type value precision_at_1000 0.11399999999999999 type value precision_at_3 12.076 type value precision_at_5 8.392 type value recall_at_1 19.945 type value recall_at_10 43.62 type value recall_at_100 67.194 type value recall_at_1000 85.7 type value recall_at_3 32.15 type value recall_at_5 37.208999999999996 task dataset metrics type Retrieval type name config split revision climate-fever MTEB ClimateFEVER default test None type value map_at_1 18.279 type value map_at_10 31.052999999999997 type value map_at_100 33.125 type value map_at_1000 33.306000000000004 type value map_at_3 26.208 type value map_at_5 28.857 type value mrr_at_1 42.671 type value mrr_at_10 54.557 type value mrr_at_100 55.142 type value mrr_at_1000 55.169000000000004 type value mrr_at_3 51.488 type value mrr_at_5 53.439 type value ndcg_at_1 42.671 type value ndcg_at_10 41.276 type value ndcg_at_100 48.376000000000005 type value ndcg_at_1000 51.318 type value ndcg_at_3 35.068 type value ndcg_at_5 37.242 type value precision_at_1 42.671 type value precision_at_10 12.638 type value precision_at_100 2.045 type value precision_at_1000 0.26 type value precision_at_3 26.08 type value precision_at_5 19.805 type value recall_at_1 18.279 type value recall_at_10 46.946 type value recall_at_100 70.97200000000001 type value recall_at_1000 87.107 type value recall_at_3 31.147999999999996 type value recall_at_5 38.099 task dataset metrics type Retrieval type name config split revision dbpedia-entity MTEB DBPedia default test None type value map_at_1 8.573 type value map_at_10 19.747 type value map_at_100 28.205000000000002 type value map_at_1000 29.831000000000003 type value map_at_3 14.109 type value map_at_5 16.448999999999998 type value mrr_at_1 71 type value mrr_at_10 77.68599999999999 type value mrr_at_100 77.995 type value mrr_at_1000 78.00200000000001 type value mrr_at_3 76.292 type value mrr_at_5 77.029 type value ndcg_at_1 59.12500000000001 type value ndcg_at_10 43.9 type value ndcg_at_100 47.863 type value ndcg_at_1000 54.848 type value ndcg_at_3 49.803999999999995 type value ndcg_at_5 46.317 type value precision_at_1 71 type value precision_at_10 34.4 type value precision_at_100 11.063 type value precision_at_1000 1.989 type value precision_at_3 52.333 type value precision_at_5 43.7 type value recall_at_1 8.573 type value recall_at_10 25.615 type value recall_at_100 53.385000000000005 type value recall_at_1000 75.46000000000001 type value recall_at_3 15.429 type value recall_at_5 19.357 task dataset metrics type Classification type name config split revision mteb/emotion MTEB EmotionClassification default test 4f58c6b202a23cf9a4da393831edf4f9183cad37 type value accuracy 47.989999999999995 type value f1 42.776314451497555 task dataset metrics type Retrieval type name config split revision fever MTEB FEVER default test None type value map_at_1 74.13499999999999 type value map_at_10 82.825 type value map_at_100 83.096 type value map_at_1000 83.111 type value map_at_3 81.748 type value map_at_5 82.446 type value mrr_at_1 79.553 type value mrr_at_10 86.654 type value mrr_at_100 86.774 type value mrr_at_1000 86.778 type value mrr_at_3 85.981 type value mrr_at_5 86.462 type value ndcg_at_1 79.553 type value ndcg_at_10 86.345 type value ndcg_at_100 87.32 type value ndcg_at_1000 87.58200000000001 type value ndcg_at_3 84.719 type value ndcg_at_5 85.677 type value precision_at_1 79.553 type value precision_at_10 10.402000000000001 type value precision_at_100 1.1119999999999999 type value precision_at_1000 0.11499999999999999 type value precision_at_3 32.413 type value precision_at_5 20.138 type value recall_at_1 74.13499999999999 type value recall_at_10 93.215 type value recall_at_100 97.083 type value recall_at_1000 98.732 type value recall_at_3 88.79 type value recall_at_5 91.259 task dataset metrics type Retrieval type name config split revision fiqa MTEB FiQA2018 default test None type value map_at_1 18.298000000000002 type value map_at_10 29.901 type value map_at_100 31.528 type value map_at_1000 31.713 type value map_at_3 25.740000000000002 type value map_at_5 28.227999999999998 type value mrr_at_1 36.728 type value mrr_at_10 45.401 type value mrr_at_100 46.27 type value mrr_at_1000 46.315 type value mrr_at_3 42.978 type value mrr_at_5 44.29 type value ndcg_at_1 36.728 type value ndcg_at_10 37.456 type value ndcg_at_100 43.832 type value ndcg_at_1000 47 type value ndcg_at_3 33.694 type value ndcg_at_5 35.085 type value precision_at_1 36.728 type value precision_at_10 10.386 type value precision_at_100 1.701 type value precision_at_1000 0.22599999999999998 type value precision_at_3 22.479 type value precision_at_5 16.605 type value recall_at_1 18.298000000000002 type value recall_at_10 44.369 type value recall_at_100 68.098 type value recall_at_1000 87.21900000000001 type value recall_at_3 30.215999999999998 type value recall_at_5 36.861 task dataset metrics type Retrieval type name config split revision hotpotqa MTEB HotpotQA default test None type value map_at_1 39.568 type value map_at_10 65.061 type value map_at_100 65.896 type value map_at_1000 65.95100000000001 type value map_at_3 61.831 type value map_at_5 63.849000000000004 type value mrr_at_1 79.136 type value mrr_at_10 84.58200000000001 type value mrr_at_100 84.765 type value mrr_at_1000 84.772 type value mrr_at_3 83.684 type value mrr_at_5 84.223 type value ndcg_at_1 79.136 type value ndcg_at_10 72.622 type value ndcg_at_100 75.539 type value ndcg_at_1000 76.613 type value ndcg_at_3 68.065 type value ndcg_at_5 70.58 type value precision_at_1 79.136 type value precision_at_10 15.215 type value precision_at_100 1.7500000000000002 type value precision_at_1000 0.189 type value precision_at_3 44.011 type value precision_at_5 28.388999999999996 type value recall_at_1 39.568 type value recall_at_10 76.077 type value recall_at_100 87.481 type value recall_at_1000 94.56400000000001 type value recall_at_3 66.01599999999999 type value recall_at_5 70.97200000000001 task dataset metrics type Classification type name config split revision mteb/imdb MTEB ImdbClassification default test 3d86128a09e091d6018b6d26cad27f2739fc2db7 type value accuracy 85.312 type value ap 80.36296867333715 type value f1 85.26613311552218 task dataset metrics type Retrieval type name config split revision msmarco MTEB MSMARCO default dev None type value map_at_1 23.363999999999997 type value map_at_10 35.711999999999996 type value map_at_100 36.876999999999995 type value map_at_1000 36.923 type value map_at_3 32.034 type value map_at_5 34.159 type value mrr_at_1 24.04 type value mrr_at_10 36.345 type value mrr_at_100 37.441 type value mrr_at_1000 37.480000000000004 type value mrr_at_3 32.713 type value mrr_at_5 34.824 type value ndcg_at_1 24.026 type value ndcg_at_10 42.531 type value ndcg_at_100 48.081 type value ndcg_at_1000 49.213 type value ndcg_at_3 35.044 type value ndcg_at_5 38.834 type value precision_at_1 24.026 type value precision_at_10 6.622999999999999 type value precision_at_100 0.941 type value precision_at_1000 0.104 type value precision_at_3 14.909 type value precision_at_5 10.871 type value recall_at_1 23.363999999999997 type value recall_at_10 63.426 type value recall_at_100 88.96300000000001 type value recall_at_1000 97.637 type value recall_at_3 43.095 type value recall_at_5 52.178000000000004 task dataset metrics type Classification type name config split revision mteb/mtop_domain MTEB MTOPDomainClassification (en) en test d80d48c1eb48d3562165c59d59d0034df9fff0bf type value accuracy 93.0095759233926 type value f1 92.78387794667408 task dataset metrics type Classification type name config split revision mteb/mtop_intent MTEB MTOPIntentClassification (en) en test ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba type value accuracy 75.0296397628819 type value f1 58.45699589820874 task dataset metrics type Classification type name config split revision mteb/amazon_massive_intent MTEB MassiveIntentClassification (en) en test 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 type value accuracy 73.45662407531944 type value f1 71.42364781421813 task dataset metrics type Classification type name config split revision mteb/amazon_massive_scenario MTEB MassiveScenarioClassification (en) en test 7d571f92784cd94a019292a1f45445077d0ef634 type value accuracy 77.07800941492937 type value f1 77.22799045640845 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-p2p MTEB MedrxivClusteringP2P default test e7a26af6f3ae46b30dde8737f02c07b1505bcc73 type value v_measure 34.531234379250606 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-s2s MTEB MedrxivClusteringS2S default test 35191c8c0dca72d8ff3efcd72aa802307d469663 type value v_measure 30.941490381193802 task dataset metrics type Reranking type name config split revision mteb/mind_small MTEB MindSmallReranking default test 3bdac13927fdc888b903db93b2ffdbd90b295a69 type value map 30.3115090856725 type value mrr 31.290667638675757 task dataset metrics type Retrieval type name config split revision nfcorpus MTEB NFCorpus default test None type value map_at_1 5.465 type value map_at_10 13.03 type value map_at_100 16.057 type value map_at_1000 17.49 type value map_at_3 9.553 type value map_at_5 11.204 type value mrr_at_1 43.653 type value mrr_at_10 53.269 type value mrr_at_100 53.72 type value mrr_at_1000 53.761 type value mrr_at_3 50.929 type value mrr_at_5 52.461 type value ndcg_at_1 42.26 type value ndcg_at_10 34.673 type value ndcg_at_100 30.759999999999998 type value ndcg_at_1000 39.728 type value ndcg_at_3 40.349000000000004 type value ndcg_at_5 37.915 type value precision_at_1 43.653 type value precision_at_10 25.789 type value precision_at_100 7.754999999999999 type value precision_at_1000 2.07 type value precision_at_3 38.596000000000004 type value precision_at_5 33.251 type value recall_at_1 5.465 type value recall_at_10 17.148 type value recall_at_100 29.768 type value recall_at_1000 62.239 type value recall_at_3 10.577 type value recall_at_5 13.315 task dataset metrics type Retrieval type name config split revision nq MTEB NQ default test None type value map_at_1 37.008 type value map_at_10 52.467 type value map_at_100 53.342999999999996 type value map_at_1000 53.366 type value map_at_3 48.412 type value map_at_5 50.875 type value mrr_at_1 41.541 type value mrr_at_10 54.967 type value mrr_at_100 55.611 type value mrr_at_1000 55.627 type value mrr_at_3 51.824999999999996 type value mrr_at_5 53.763000000000005 type value ndcg_at_1 41.541 type value ndcg_at_10 59.724999999999994 type value ndcg_at_100 63.38700000000001 type value ndcg_at_1000 63.883 type value ndcg_at_3 52.331 type value ndcg_at_5 56.327000000000005 type value precision_at_1 41.541 type value precision_at_10 9.447 type value precision_at_100 1.1520000000000001 type value precision_at_1000 0.12 type value precision_at_3 23.262 type value precision_at_5 16.314999999999998 type value recall_at_1 37.008 type value recall_at_10 79.145 type value recall_at_100 94.986 type value recall_at_1000 98.607 type value recall_at_3 60.277 type value recall_at_5 69.407 task dataset metrics type Retrieval type name config split revision quora MTEB QuoraRetrieval default test None type value map_at_1 70.402 type value map_at_10 84.181 type value map_at_100 84.796 type value map_at_1000 84.81400000000001 type value map_at_3 81.209 type value map_at_5 83.085 type value mrr_at_1 81.02000000000001 type value mrr_at_10 87.263 type value mrr_at_100 87.36 type value mrr_at_1000 87.36 type value mrr_at_3 86.235 type value mrr_at_5 86.945 type value ndcg_at_1 81.01 type value ndcg_at_10 87.99900000000001 type value ndcg_at_100 89.217 type value ndcg_at_1000 89.33 type value ndcg_at_3 85.053 type value ndcg_at_5 86.703 type value precision_at_1 81.01 type value precision_at_10 13.336 type value precision_at_100 1.52 type value precision_at_1000 0.156 type value precision_at_3 37.14 type value precision_at_5 24.44 type value recall_at_1 70.402 type value recall_at_10 95.214 type value recall_at_100 99.438 type value recall_at_1000 99.928 type value recall_at_3 86.75699999999999 type value recall_at_5 91.44099999999999 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering MTEB RedditClustering default test 24640382cdbf8abc73003fb0fa6d111a705499eb type value v_measure 56.51721502758904 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering-p2p MTEB RedditClusteringP2P default test 282350215ef01743dc01b456c7f5241fa8937f16 type value v_measure 61.054808572333016 task dataset metrics type Retrieval type name config split revision scidocs MTEB SCIDOCS default test None type value map_at_1 4.578 type value map_at_10 11.036999999999999 type value map_at_100 12.879999999999999 type value map_at_1000 13.150999999999998 type value map_at_3 8.133 type value map_at_5 9.559 type value mrr_at_1 22.6 type value mrr_at_10 32.68 type value mrr_at_100 33.789 type value mrr_at_1000 33.854 type value mrr_at_3 29.7 type value mrr_at_5 31.480000000000004 type value ndcg_at_1 22.6 type value ndcg_at_10 18.616 type value ndcg_at_100 25.883 type value ndcg_at_1000 30.944 type value ndcg_at_3 18.136 type value ndcg_at_5 15.625 type value precision_at_1 22.6 type value precision_at_10 9.48 type value precision_at_100 1.991 type value precision_at_1000 0.321 type value precision_at_3 16.8 type value precision_at_5 13.54 type value recall_at_1 4.578 type value recall_at_10 19.213 type value recall_at_100 40.397 type value recall_at_1000 65.2 type value recall_at_3 10.208 type value recall_at_5 13.718 task dataset metrics type STS type name config split revision mteb/sickr-sts MTEB SICK-R default test a6ea5a8cab320b040a23452cc28066d9beae2cee type value cos_sim_pearson 83.44288351714071 type value cos_sim_spearman 79.37995604564952 type value euclidean_pearson 81.1078874670718 type value euclidean_spearman 79.37995905980499 type value manhattan_pearson 81.03697527288986 type value manhattan_spearman 79.33490235296236 task dataset metrics type STS type name config split revision mteb/sts12-sts MTEB STS12 default test a0d554a64d88156834ff5ae9920b964011b16384 type value cos_sim_pearson 84.95557650436523 type value cos_sim_spearman 78.5190672399868 type value euclidean_pearson 81.58064025904707 type value euclidean_spearman 78.5190672399868 type value manhattan_pearson 81.52857930619889 type value manhattan_spearman 78.50421361308034 task dataset metrics type STS type name config split revision mteb/sts13-sts MTEB STS13 default test 7e90230a92c190f1bf69ae9002b8cea547a64cca type value cos_sim_pearson 84.79128416228737 type value cos_sim_spearman 86.05402451477147 type value euclidean_pearson 85.46280267054289 type value euclidean_spearman 86.05402451477147 type value manhattan_pearson 85.46278563858236 type value manhattan_spearman 86.08079590861004 task dataset metrics type STS type name config split revision mteb/sts14-sts MTEB STS14 default test 6031580fec1f6af667f0bd2da0a551cf4f0b2375 type value cos_sim_pearson 83.20623089568763 type value cos_sim_spearman 81.53786907061009 type value euclidean_pearson 82.82272250091494 type value euclidean_spearman 81.53786907061009 type value manhattan_pearson 82.78850494027013 type value manhattan_spearman 81.5135618083407 task dataset metrics type STS type name config split revision mteb/sts15-sts MTEB STS15 default test ae752c7c21bf194d8b67fd573edf7ae58183cbe3 type value cos_sim_pearson 85.46366618397936 type value cos_sim_spearman 86.96566013336908 type value euclidean_pearson 86.62651697548931 type value euclidean_spearman 86.96565526364454 type value manhattan_pearson 86.58812160258009 type value manhattan_spearman 86.9336484321288 task dataset metrics type STS type name config split revision mteb/sts16-sts MTEB STS16 default test 4d8694f8f0e0100860b497b999b3dbed754a0513 type value cos_sim_pearson 82.51858358641559 type value cos_sim_spearman 84.7652527954999 type value euclidean_pearson 84.23914783766861 type value euclidean_spearman 84.7652527954999 type value manhattan_pearson 84.22749648503171 type value manhattan_spearman 84.74527996746386 task dataset metrics type STS type name config split revision mteb/sts17-crosslingual-sts MTEB STS17 (en-en) en-en test af5e6fb845001ecf41f4c1e033ce921939a2a68d type value cos_sim_pearson 87.28026563313065 type value cos_sim_spearman 87.46928143824915 type value euclidean_pearson 88.30558762000372 type value euclidean_spearman 87.46928143824915 type value manhattan_pearson 88.10513330809331 type value manhattan_spearman 87.21069787834173 task dataset metrics type STS type name config split revision mteb/sts22-crosslingual-sts MTEB STS22 (en) en test 6d1ba47164174a496b7fa5d3569dae26a6813b80 type value cos_sim_pearson 62.376497134587375 type value cos_sim_spearman 65.0159550112516 type value euclidean_pearson 65.64572120879598 type value euclidean_spearman 65.0159550112516 type value manhattan_pearson 65.88143604989976 type value manhattan_spearman 65.17547297222434 task dataset metrics type STS type name config split revision mteb/stsbenchmark-sts MTEB STSBenchmark default test b0fddb56ed78048fa8b90373c8a3cfc37b684831 type value cos_sim_pearson 84.22876368947644 type value cos_sim_spearman 85.46935577445318 type value euclidean_pearson 85.32830231392005 type value euclidean_spearman 85.46935577445318 type value manhattan_pearson 85.30353211758495 type value manhattan_spearman 85.42821085956945 task dataset metrics type Reranking type name config split revision mteb/scidocs-reranking MTEB SciDocsRR default test d3c5e1fc0b855ab6097bf1cda04dd73947d7caab type value map 80.60986667767133 type value mrr 94.29432314236236 task dataset metrics type Retrieval type name config split revision scifact MTEB SciFact default test None type value map_at_1 54.528 type value map_at_10 65.187 type value map_at_100 65.62599999999999 type value map_at_1000 65.657 type value map_at_3 62.352 type value map_at_5 64.025 type value mrr_at_1 57.333 type value mrr_at_10 66.577 type value mrr_at_100 66.88 type value mrr_at_1000 66.908 type value mrr_at_3 64.556 type value mrr_at_5 65.739 type value ndcg_at_1 57.333 type value ndcg_at_10 70.275 type value ndcg_at_100 72.136 type value ndcg_at_1000 72.963 type value ndcg_at_3 65.414 type value ndcg_at_5 67.831 type value precision_at_1 57.333 type value precision_at_10 9.5 type value precision_at_100 1.057 type value precision_at_1000 0.11199999999999999 type value precision_at_3 25.778000000000002 type value precision_at_5 17.2 type value recall_at_1 54.528 type value recall_at_10 84.356 type value recall_at_100 92.833 type value recall_at_1000 99.333 type value recall_at_3 71.283 type value recall_at_5 77.14999999999999 task dataset metrics type PairClassification type name config split revision mteb/sprintduplicatequestions-pairclassification MTEB SprintDuplicateQuestions default test d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 type value cos_sim_accuracy 99.74158415841585 type value cos_sim_ap 92.90048959850317 type value cos_sim_f1 86.35650810245687 type value cos_sim_precision 90.4709748083242 type value cos_sim_recall 82.6 type value dot_accuracy 99.74158415841585 type value dot_ap 92.90048959850317 type value dot_f1 86.35650810245687 type value dot_precision 90.4709748083242 type value dot_recall 82.6 type value euclidean_accuracy 99.74158415841585 type value euclidean_ap 92.90048959850317 type value euclidean_f1 86.35650810245687 type value euclidean_precision 90.4709748083242 type value euclidean_recall 82.6 type value manhattan_accuracy 99.74158415841585 type value manhattan_ap 92.87344692947894 type value manhattan_f1 86.38497652582159 type value manhattan_precision 90.29443838604145 type value manhattan_recall 82.8 type value max_accuracy 99.74158415841585 type value max_ap 92.90048959850317 type value max_f1 86.38497652582159 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering MTEB StackExchangeClustering default test 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 type value v_measure 63.191648770424216 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering-p2p MTEB StackExchangeClusteringP2P default test 815ca46b2622cec33ccafc3735d572c266efdb44 type value v_measure 34.02944668730218 task dataset metrics type Reranking type name config split revision mteb/stackoverflowdupquestions-reranking MTEB StackOverflowDupQuestions default test e185fbe320c72810689fc5848eb6114e1ef5ec69 type value map 50.466386167525265 type value mrr 51.19071492233257 task dataset metrics type Summarization type name config split revision mteb/summeval MTEB SummEval default test cda12ad7615edc362dbf25a00fdd61d3b1eaf93c type value cos_sim_pearson 30.198022505886435 type value cos_sim_spearman 30.40170257939193 type value dot_pearson 30.198015316402614 type value dot_spearman 30.40170257939193 task dataset metrics type Retrieval type name config split revision trec-covid MTEB TRECCOVID default test None type value map_at_1 0.242 type value map_at_10 2.17 type value map_at_100 12.221 type value map_at_1000 28.63 type value map_at_3 0.728 type value map_at_5 1.185 type value mrr_at_1 94 type value mrr_at_10 97 type value mrr_at_100 97 type value mrr_at_1000 97 type value mrr_at_3 97 type value mrr_at_5 97 type value ndcg_at_1 89 type value ndcg_at_10 82.30499999999999 type value ndcg_at_100 61.839999999999996 type value ndcg_at_1000 53.381 type value ndcg_at_3 88.877 type value ndcg_at_5 86.05199999999999 type value precision_at_1 94 type value precision_at_10 87 type value precision_at_100 63.38 type value precision_at_1000 23.498 type value precision_at_3 94 type value precision_at_5 92 type value recall_at_1 0.242 type value recall_at_10 2.302 type value recall_at_100 14.979000000000001 type value recall_at_1000 49.638 type value recall_at_3 0.753 type value recall_at_5 1.226 task dataset metrics type Retrieval type name config split revision webis-touche2020 MTEB Touche2020 default test None type value map_at_1 3.006 type value map_at_10 11.805 type value map_at_100 18.146 type value map_at_1000 19.788 type value map_at_3 5.914 type value map_at_5 8.801 type value mrr_at_1 40.816 type value mrr_at_10 56.36600000000001 type value mrr_at_100 56.721999999999994 type value mrr_at_1000 56.721999999999994 type value mrr_at_3 52.041000000000004 type value mrr_at_5 54.796 type value ndcg_at_1 37.755 type value ndcg_at_10 29.863 type value ndcg_at_100 39.571 type value ndcg_at_1000 51.385999999999996 type value ndcg_at_3 32.578 type value ndcg_at_5 32.351 type value precision_at_1 40.816 type value precision_at_10 26.531 type value precision_at_100 7.796 type value precision_at_1000 1.555 type value precision_at_3 32.653 type value precision_at_5 33.061 type value recall_at_1 3.006 type value recall_at_10 18.738 type value recall_at_100 48.058 type value recall_at_1000 83.41300000000001 type value recall_at_3 7.166 type value recall_at_5 12.102 task dataset metrics type Classification type name config split revision mteb/toxic_conversations_50k MTEB ToxicConversationsClassification default test d7c0de2777da35d6aae2200a62c6e0e5af397c4c type value accuracy 71.4178 type value ap 14.648781342150446 type value f1 55.07299194946378 task dataset metrics type Classification type name config split revision mteb/tweet_sentiment_extraction MTEB TweetSentimentExtractionClassification default test d604517c81ca91fe16a244d1248fc021f9ecee7a type value accuracy 60.919637804187886 type value f1 61.24122013967399 task dataset metrics type Clustering type name config split revision mteb/twentynewsgroups-clustering MTEB TwentyNewsgroupsClustering default test 6125ec4e24fa026cec8a478383ee943acfbd5449 type value v_measure 49.207896583685695 task dataset metrics type PairClassification type name config split revision mteb/twittersemeval2015-pairclassification MTEB TwitterSemEval2015 default test 70970daeab8776df92f5ea462b6173c0b46fd2d1 type value cos_sim_accuracy 86.23114978840078 type value cos_sim_ap 74.26624727825818 type value cos_sim_f1 68.72377190817083 type value cos_sim_precision 64.56400742115028 type value cos_sim_recall 73.45646437994723 type value dot_accuracy 86.23114978840078 type value dot_ap 74.26624032659652 type value dot_f1 68.72377190817083 type value dot_precision 64.56400742115028 type value dot_recall 73.45646437994723 type value euclidean_accuracy 86.23114978840078 type value euclidean_ap 74.26624714480556 type value euclidean_f1 68.72377190817083 type value euclidean_precision 64.56400742115028 type value euclidean_recall 73.45646437994723 type value manhattan_accuracy 86.16558383501221 type value manhattan_ap 74.2091943976357 type value manhattan_f1 68.64221520524654 type value manhattan_precision 63.59135913591359 type value manhattan_recall 74.5646437994723 type value max_accuracy 86.23114978840078 type value max_ap 74.26624727825818 type value max_f1 68.72377190817083 task dataset metrics type PairClassification type name config split revision mteb/twitterurlcorpus-pairclassification MTEB TwitterURLCorpus default test 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf type value cos_sim_accuracy 89.3681841114604 type value cos_sim_ap 86.65166387498546 type value cos_sim_f1 79.02581944698774 type value cos_sim_precision 75.35796605434099 type value cos_sim_recall 83.06898675700647 type value dot_accuracy 89.3681841114604 type value dot_ap 86.65166019802056 type value dot_f1 79.02581944698774 type value dot_precision 75.35796605434099 type value dot_recall 83.06898675700647 type value euclidean_accuracy 89.3681841114604 type value euclidean_ap 86.65166462876266 type value euclidean_f1 79.02581944698774 type value euclidean_precision 75.35796605434099 type value euclidean_recall 83.06898675700647 type value manhattan_accuracy 89.36624364497226 type value manhattan_ap 86.65076471274106 type value manhattan_f1 79.07408783532733 type value manhattan_precision 76.41102972856527 type value manhattan_recall 81.92947336002464 type value max_accuracy 89.3681841114604 type value max_ap 86.65166462876266 type value max_f1 79.07408783532733	apache-2.0	en

nomic-embed-text-v1.5: Resizable Production Embeddings with Matryoshka Representation Learning

Blog | Technical Report | AWS SageMaker | Nomic Platform

Exciting Update!: nomic-embed-text-v1.5 is now multimodal! nomic-embed-vision-v1.5 is aligned to the embedding space of nomic-embed-text-v1.5, meaning any text embedding is multimodal!

Usage

Important: the text prompt must include a task instruction prefix, instructing the model which task is being performed.

For example, if you are implementing a RAG application, you embed your documents as search_document: <text here> and embed your user queries as search_query: <text here>.

Task instruction prefixes

search_document

Purpose: embed texts as documents from a dataset

This prefix is used for embedding texts as documents, for example as documents for a RAG index.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
sentences = ['search_document: TSNE is a dimensionality reduction algorithm created by Laurens van Der Maaten']
embeddings = model.encode(sentences)
print(embeddings)

search_query

Purpose: embed texts as questions to answer

This prefix is used for embedding texts as questions that documents from a dataset could resolve, for example as queries to be answered by a RAG application.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
sentences = ['search_query: Who is Laurens van Der Maaten?']
embeddings = model.encode(sentences)
print(embeddings)

clustering

Purpose: embed texts to group them into clusters

This prefix is used for embedding texts in order to group them into clusters, discover common topics, or remove semantic duplicates.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
sentences = ['clustering: the quick brown fox']
embeddings = model.encode(sentences)
print(embeddings)

classification

Purpose: embed texts to classify them

This prefix is used for embedding texts into vectors that will be used as features for a classification model

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
sentences = ['classification: the quick brown fox']
embeddings = model.encode(sentences)
print(embeddings)

Sentence Transformers

import torch.nn.functional as F
from sentence_transformers import SentenceTransformer

matryoshka_dim = 512

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
sentences = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?']
embeddings = model.encode(sentences, convert_to_tensor=True)
embeddings = F.layer_norm(embeddings, normalized_shape=(embeddings.shape[1],))
embeddings = embeddings[:, :matryoshka_dim]
embeddings = F.normalize(embeddings, p=2, dim=1)
print(embeddings)

Transformers

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0]
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

sentences = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?']

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = AutoModel.from_pretrained('nomic-ai/nomic-embed-text-v1.5', trust_remote_code=True, safe_serialization=True)
model.eval()

encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

+ matryoshka_dim = 512

with torch.no_grad():
    model_output = model(**encoded_input)

embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
+ embeddings = F.layer_norm(embeddings, normalized_shape=(embeddings.shape[1],))
+ embeddings = embeddings[:, :matryoshka_dim]
embeddings = F.normalize(embeddings, p=2, dim=1)
print(embeddings)

The model natively supports scaling of the sequence length past 2048 tokens. To do so,

- tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
+ tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased', model_max_length=8192)


- model = AutoModel.from_pretrained('nomic-ai/nomic-embed-text-v1.5', trust_remote_code=True)
+ model = AutoModel.from_pretrained('nomic-ai/nomic-embed-text-v1.5', trust_remote_code=True, rotary_scaling_factor=2)

Transformers.js

import { pipeline, layer_norm } from '@huggingface/transformers';

// Create a feature extraction pipeline
const extractor = await pipeline('feature-extraction', 'nomic-ai/nomic-embed-text-v1.5');

// Define sentences
const texts = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?'];

// Compute sentence embeddings
let embeddings = await extractor(texts, { pooling: 'mean' });
console.log(embeddings); // Tensor of shape [2, 768]

const matryoshka_dim = 512;
embeddings = layer_norm(embeddings, [embeddings.dims[1]])
    .slice(null, [0, matryoshka_dim])
    .normalize(2, -1);
console.log(embeddings.tolist());

Nomic API

The easiest way to use Nomic Embed is through the Nomic Embedding API.

Generating embeddings with the nomic Python client is as easy as

from nomic import embed

output = embed.text(
    texts=['Nomic Embedding API', '#keepAIOpen'],
    model='nomic-embed-text-v1.5',
    task_type='search_document',
    dimensionality=256,
)

print(output)

For more information, see the API reference

Infinity

Usage with Infinity.

docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
michaelf34/infinity:0.0.70 \
v2 --model-id nomic-ai/nomic-embed-text-v1.5 --revision "main" --dtype float16 --batch-size 8 --engine torch --port 7997 --no-bettertransformer

Adjusting Dimensionality

nomic-embed-text-v1.5 is an improvement upon Nomic Embed that utilizes Matryoshka Representation Learning which gives developers the flexibility to trade off the embedding size for a negligible reduction in performance.

Name	SeqLen	Dimension	MTEB
nomic-embed-text-v1	8192	768	62.39
nomic-embed-text-v1.5	8192	768	62.28
nomic-embed-text-v1.5	8192	512	61.96
nomic-embed-text-v1.5	8192	256	61.04
nomic-embed-text-v1.5	8192	128	59.34
nomic-embed-text-v1.5	8192	64	56.10

Training

Click the Nomic Atlas map below to visualize a 5M sample of our contrastive pretraining data!

We train our embedder using a multi-stage training pipeline. Starting from a long-context BERT model, the first unsupervised contrastive stage trains on a dataset generated from weakly related text pairs, such as question-answer pairs from forums like StackExchange and Quora, title-body pairs from Amazon reviews, and summarizations from news articles.

In the second finetuning stage, higher quality labeled datasets such as search queries and answers from web searches are leveraged. Data curation and hard-example mining is crucial in this stage.

For more details, see the Nomic Embed Technical Report and corresponding blog post.

Training data to train the models is released in its entirety. For more details, see the contrastors repository

Join the Nomic Community

• Nomic: https://nomic.ai
• Discord: https://discord.gg/myY5YDR8z8
• Twitter: https://twitter.com/nomic_ai

Citation

If you find the model, dataset, or training code useful, please cite our work

@misc{nussbaum2024nomic,
      title={Nomic Embed: Training a Reproducible Long Context Text Embedder}, 
      author={Zach Nussbaum and John X. Morris and Brandon Duderstadt and Andriy Mulyar},
      year={2024},
      eprint={2402.01613},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}