Bittensor
Search…
β˜•
Creating a Server
Pretrained models

Pretrained Models

At Bittensor, the server class is built with the huggingface API and therefore has access to all of the pretrained models currently on huggingface (Pretrained Models). Using--server.model_nameor by changing model_name argument when creating the server, we are able to load the corresponding pretrained model as well as its tokenizer from huggingface.
1
# Sample Usage (Bash)
2
python /miners/text/template_server.py --server.model_name bert-base-uncased
3
​
4
# Sample Usage (Python)
5
model = server(model_name='bert-base-uncased',pretrained=True)
Copied!
If you wish to load the same model structure as a pretrained model but not the pretrained weights, simple set pretrained = False when creating the model.
If you have a custom model and tokenizer that you wish to attach server class, you can use the model and tokenizer argument in the server class.
1
model = torch.TransformerEncoderLayer( 512,100,2,0.1)
2
tokenizer = bittensor.tokenizer()
3
​
4
server = server(model=model ,tokenizer=tokenizer )
Copied!

Padding

At Bittensor, all peer-to-peer responses should have a hidden dimension that matches the bittensor's network dimension (1024 currently). In order to ensure that all responses from pre-trained models meet that requirement, the final response is padded to match bittensor's network dimension. For pretrained models that have a larger hidden dimension than the network dimension, turning padding=False will automatically create an additional linear layer that downsamples from the larger hidden size to 1024.
1
#For large pretrained models
2
model = server(model_name='gpt2-large',pretrained=True, padding=False)
Copied!

Interpolation

To ensure that the sequence length is maintained across different tokenizers, the server class contains the option to interpolate between sequence lengths. This uses torch's interpolation function and contains several possible algorithms for interpolation. It is possible to turn off the default interpolation function and add your own custom mapping function that maps between tokenizers.
1
# interpolation online, bilinear algorithm
2
model = server(model_name='gpt2',pretrained=True, interpolate=True, inter_degree=''bilinear)
3
​
4
# custom mapping function
5
model = server(model_name='gpt2',pretrained=True, interpolate=False, mapping_function=(..))
Copied!
Last modified 20d ago