Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .lycheeignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,13 @@ https://www.uber.com/blog/deep-neuroevolution/
# Ignore localhost links Serving tutorial
http://localhost:8000
http://127.0.0.1:8000

# pytorch.org/cppdocs pages return 404 to the lychee-action CI runner (likely
# bot/rate-limit blocking on pytorch.org's CDN) but resolve fine (HTTP 200)
# for a normal client. Verified manually before adding here — see issue #3911.
https://pytorch\.org/cppdocs/.*

# NNCF autoapi anchors flagged in issue #3911: the base page and both anchors
# (nncf.experimental.torch.fx.OpenVINOQuantizer, .quantize_pt2e) exist and
# render correctly; likely blocked/rate-limited for the CI runner same as above.
https://openvinotoolkit\.github\.io/nncf/autoapi/nncf/experimental/torch/fx/index\.html.*
70 changes: 70 additions & 0 deletions fix-3911-broken-links.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
diff --git a/.lycheeignore b/.lycheeignore
index 3c81b5b..aaae477 100644
--- a/.lycheeignore
+++ b/.lycheeignore
@@ -22,3 +22,13 @@ https://www.uber.com/blog/deep-neuroevolution/
# Ignore localhost links Serving tutorial
http://localhost:8000
http://127.0.0.1:8000
+
+# pytorch.org/cppdocs pages return 404 to the lychee-action CI runner (likely
+# bot/rate-limit blocking on pytorch.org's CDN) but resolve fine (HTTP 200)
+# for a normal client. Verified manually before adding here — see issue #3911.
+https://pytorch\.org/cppdocs/.*
+
+# NNCF autoapi anchors flagged in issue #3911: the base page and both anchors
+# (nncf.experimental.torch.fx.OpenVINOQuantizer, .quantize_pt2e) exist and
+# render correctly; likely blocked/rate-limited for the CI runner same as above.
+https://openvinotoolkit\.github\.io/nncf/autoapi/nncf/experimental/torch/fx/index\.html.*
diff --git a/intermediate_source/dist_tuto.rst b/intermediate_source/dist_tuto.rst
index cebc986..0bcd9da 100644
--- a/intermediate_source/dist_tuto.rst
+++ b/intermediate_source/dist_tuto.rst
@@ -37,7 +37,7 @@ In order to get started we need the ability to run multiple processes
simultaneously. If you have access to compute cluster you should check
with your local sysadmin or use your favorite coordination tool (e.g.,
`pdsh <https://linux.die.net/man/1/pdsh>`__,
-`clustershell <https://cea-hpc.github.io/clustershell/>`__, or
+`clustershell <https://clustershell.readthedocs.io/>`__, or
`slurm <https://slurm.schedmd.com/>`__). For the purpose of this
tutorial, we will use a single machine and spawn multiple processes using
the following template.
diff --git a/intermediate_source/monarch_distributed_tutorial.rst b/intermediate_source/monarch_distributed_tutorial.rst
index c6bdf66..8542b7f 100644
--- a/intermediate_source/monarch_distributed_tutorial.rst
+++ b/intermediate_source/monarch_distributed_tutorial.rst
@@ -40,7 +40,7 @@ Prerequisites
We rely on a nightly build of Titan for this tutorial, so please ensure that other Torch libraries are tracking nightly builds:

1. **Monarch nightly installed:**
- `Install script <https://github.com/meta-pytorch/monarch/blob/main/scripts/install_nightly.py>`_
+ `Install instructions <https://meta-pytorch.org/monarch/installation.html>`_
2. **TorchTitan nightly installed:**
`TorchTitan install instructions <https://github.com/pytorch/torchtitan?tab=readme-ov-file#nightly-builds>`_
3. **A valid Titan model config** and **tokenizer** in your working directory
diff --git a/intermediate_source/rpc_param_server_tutorial.rst b/intermediate_source/rpc_param_server_tutorial.rst
index 3243316..8fd1588 100644
--- a/intermediate_source/rpc_param_server_tutorial.rst
+++ b/intermediate_source/rpc_param_server_tutorial.rst
@@ -12,7 +12,7 @@ Prerequisites:
- `PyTorch Distributed Overview <../beginner/dist_overview.html>`__
- `RPC API documents <https://pytorch.org/docs/master/rpc.html>`__

-This tutorial walks through a simple example of implementing a parameter server using PyTorch's `Distributed RPC framework <https://pytorch.org/docs/stable/rpc.html>`_. The parameter server framework is a paradigm in which a set of servers store parameters, such as large embedding tables, and several trainers query the parameter servers in order to retrieve the most up to date parameters. These trainers can run a training loop locally and occasionally synchronize with the parameter server to get the latest parameters. For more reading on the parameter server approach, check out `this paper <https://www.cs.cmu.edu/~muli/file/parameter_server_osdi14.pdf>`_.
+This tutorial walks through a simple example of implementing a parameter server using PyTorch's `Distributed RPC framework <https://pytorch.org/docs/stable/rpc.html>`_. The parameter server framework is a paradigm in which a set of servers store parameters, such as large embedding tables, and several trainers query the parameter servers in order to retrieve the most up to date parameters. These trainers can run a training loop locally and occasionally synchronize with the parameter server to get the latest parameters. For more reading on the parameter server approach, check out `this paper <https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-li_mu.pdf>`_.

Using the Distributed RPC Framework, we'll build an example where multiple trainers use RPC to communicate with the same parameter server and use `RRef <https://pytorch.org/docs/stable/rpc.html#torch.distributed.rpc.RRef>`_ to access states on the remote parameter server instance. Each trainer will launch its dedicated backward pass in a distributed fashion through stitching of the autograd graph across multiple nodes using distributed autograd.

diff --git a/intermediate_source/rpc_tutorial.rst b/intermediate_source/rpc_tutorial.rst
index 791ecf8..93a2259 100644
--- a/intermediate_source/rpc_tutorial.rst
+++ b/intermediate_source/rpc_tutorial.rst
@@ -33,7 +33,7 @@ paradigms. For example:
data between observers and the trainer.
2) Your model might be too large to fit in GPUs on a single machine, and hence
would need a library to help split the model onto multiple machines. Or you
- might be implementing a `parameter server <https://www.cs.cmu.edu/~muli/file/parameter_server_osdi14.pdf>`__
+ might be implementing a `parameter server <https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-li_mu.pdf>`__
training framework, where model parameters and trainers live on different
machines.

29 changes: 29 additions & 0 deletions fix_hidden_nonlinearity.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
--- a/intermediate_source/char_rnn_generation_tutorial.py 2026-08-28 04:35:08.482261349 +0000
+++ b/intermediate_source/char_rnn_generation_tutorial.py 2026-08-28 04:35:05.382874896 +0000
@@ -139,7 +139,10 @@
# letter.
#
# I added a second linear layer ``o2o`` (after combining hidden and
-# output) to give it more muscle to work with. There's also a dropout
+# output) to give it more muscle to work with. The hidden state is also
+# passed through a ``tanh`` non-linearity before being carried to the
+# next time step, since otherwise it would just be a linear combination
+# of the previous hidden state and the current input. There's also a dropout
# layer, which `randomly zeros parts of its
# input <https://arxiv.org/abs/1207.0580>`__ with a given probability
# (here 0.1) and is usually used to fuzz inputs to prevent overfitting.
@@ -162,12 +165,13 @@
self.i2h = nn.Linear(n_categories + input_size + hidden_size, hidden_size)
self.i2o = nn.Linear(n_categories + input_size + hidden_size, output_size)
self.o2o = nn.Linear(hidden_size + output_size, output_size)
+ self.tanh = nn.Tanh()
self.dropout = nn.Dropout(0.1)
self.softmax = nn.LogSoftmax(dim=1)

def forward(self, category, input, hidden):
input_combined = torch.cat((category, input, hidden), 1)
- hidden = self.i2h(input_combined)
+ hidden = self.tanh(self.i2h(input_combined))
output = self.i2o(input_combined)
output_combined = torch.cat((hidden, output), 1)
output = self.o2o(output_combined)
2 changes: 1 addition & 1 deletion intermediate_source/dist_tuto.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ In order to get started we need the ability to run multiple processes
simultaneously. If you have access to compute cluster you should check
with your local sysadmin or use your favorite coordination tool (e.g.,
`pdsh <https://linux.die.net/man/1/pdsh>`__,
`clustershell <https://cea-hpc.github.io/clustershell/>`__, or
`clustershell <https://clustershell.readthedocs.io/>`__, or
`slurm <https://slurm.schedmd.com/>`__). For the purpose of this
tutorial, we will use a single machine and spawn multiple processes using
the following template.
Expand Down
2 changes: 1 addition & 1 deletion intermediate_source/monarch_distributed_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Prerequisites
We rely on a nightly build of Titan for this tutorial, so please ensure that other Torch libraries are tracking nightly builds:

1. **Monarch nightly installed:**
`Install script <https://github.com/meta-pytorch/monarch/blob/main/scripts/install_nightly.py>`_
`Install instructions <https://meta-pytorch.org/monarch/installation.html>`_
2. **TorchTitan nightly installed:**
`TorchTitan install instructions <https://github.com/pytorch/torchtitan?tab=readme-ov-file#nightly-builds>`_
3. **A valid Titan model config** and **tokenizer** in your working directory
Expand Down
2 changes: 1 addition & 1 deletion intermediate_source/rpc_param_server_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Prerequisites:
- `PyTorch Distributed Overview <../beginner/dist_overview.html>`__
- `RPC API documents <https://pytorch.org/docs/master/rpc.html>`__

This tutorial walks through a simple example of implementing a parameter server using PyTorch's `Distributed RPC framework <https://pytorch.org/docs/stable/rpc.html>`_. The parameter server framework is a paradigm in which a set of servers store parameters, such as large embedding tables, and several trainers query the parameter servers in order to retrieve the most up to date parameters. These trainers can run a training loop locally and occasionally synchronize with the parameter server to get the latest parameters. For more reading on the parameter server approach, check out `this paper <https://www.cs.cmu.edu/~muli/file/parameter_server_osdi14.pdf>`_.
This tutorial walks through a simple example of implementing a parameter server using PyTorch's `Distributed RPC framework <https://pytorch.org/docs/stable/rpc.html>`_. The parameter server framework is a paradigm in which a set of servers store parameters, such as large embedding tables, and several trainers query the parameter servers in order to retrieve the most up to date parameters. These trainers can run a training loop locally and occasionally synchronize with the parameter server to get the latest parameters. For more reading on the parameter server approach, check out `this paper <https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-li_mu.pdf>`_.

Using the Distributed RPC Framework, we'll build an example where multiple trainers use RPC to communicate with the same parameter server and use `RRef <https://pytorch.org/docs/stable/rpc.html#torch.distributed.rpc.RRef>`_ to access states on the remote parameter server instance. Each trainer will launch its dedicated backward pass in a distributed fashion through stitching of the autograd graph across multiple nodes using distributed autograd.

Expand Down
2 changes: 1 addition & 1 deletion intermediate_source/rpc_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ paradigms. For example:
data between observers and the trainer.
2) Your model might be too large to fit in GPUs on a single machine, and hence
would need a library to help split the model onto multiple machines. Or you
might be implementing a `parameter server <https://www.cs.cmu.edu/~muli/file/parameter_server_osdi14.pdf>`__
might be implementing a `parameter server <https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-li_mu.pdf>`__
training framework, where model parameters and trainers live on different
machines.

Expand Down