In theoretical physics, a 't Hooft operator, introduced by Gerard 't Hooft in the 1978 paper "On the phase transition towards permanent quark confinement", is a dual version of the Wilson loop in which the electromagnetic potential A is replaced by its electromagnetic dual Amag, where the exterior derivative of A is equal to the Hodge dual of the exterior derivative of Amag. In d spacetime dimensions, Amag is a (d-3)-form and so the 't Hooft operator is the integral of Amag over a (d-3)-dimensional surface.
While the Wilson loop is an order operator, the 't Hooft operator is an example of a disorder operator because it creates a singularity or a discontinuity in the fundamental fields such as the electromagnetic potential A. For example, in an SU(N) Yang Mills gauge theory a 't Hooft operator creates a Dirac magnetic monopole with respect to the center of SU(N). If a condensate is present which transforms in a representation of SU(N) which is invariant under the action of the center, such as the adjoint representation, then the magnetic monopole will be confined by a vortex lying along a Dirac string from the monopole to either an antimonopole or to infinity. This vortex is similar to a Nielsen-Olesen vortex, but it carries a charge under the center of SU(N), and so N such vortices may annihilate.
In his 1978 paper, 't Hooft demonstrated that Wilson loops and 't Hooft operators commute up to a phase which is an n-th root of unity. The choice of root of unity is related to the linking number of the Wilson loop and the vortex. 't Hooft claimed that this apparently non-local commutation relation implies that any phase of a Yang-Mills gauge theory must either contain massless particles, responsible for the interactions between the 't Hooft operator and the Wilson loop, or else at least one of the two operators must be confined by an object one dimension higher. He identified the phase in which the 't Hooft operator is confined as the Higgs phase, in which the confinement of magnetic monopoles by vortices was a well-known consequence of the Meissner effect, already observed in type II superconductors. He identified the phase in which the Wilson loop is confined as the confining phase, as a Wilson loop is the action of an electric charge. Finally he left open the possibility of mixed phases, in which both are confined. Although such mixed phases had not been seen in quantum field theory at the time, they are now know to occur for example in the Argyres-Douglas conformal field theory. Therefore, he argued that gauge theories are necessarily in one of these four possible phases.
't Hooft found a simple formula for the scalings of the Wilson and 't Hooft operators in the various phases. When a given operator is confined, a finite tension surface is created whose boundary is the operator. The action of the configuration, in the limit in which the configuration is large, therefore scales with the volume of this surface. In the confining phase the Wilson loops are confined by a 2-dimensional surface, and so the action of a Wilson loop scales as the area of this surface. In the Higgs phase the (d-3)-dimensional 't Hooft operator is confined, and so the action scales as the area of the (d-2)-dimensional confining surface. For example, in the confining phase in 4 space-time dimensions the action of the 't Hooft operator scales as the distance squared. In the mixed phase both operators are confined, and so both obey this scaling.
On the other hand, he claimed that if a given operator is Higgsed, then the corresponding gluons are massive and so the action falls off exponentially away from the operator. Therefore, the action will be proportional to volume of the surface on which the operator is evaluated itself. For example, in the Higgs phase the gauge field is Higgsed and so the Wilson loop action is proportional to the length of the loop, which scales linearly with distance. In the confining phase the 't Hooft operator is Higgsed, and so the corresponding action fails as the area of the corresponding (d-3)-dimensional surface, for example linearly in 4 spacetime dimensions. In particular 't Hooft concluded that in 4 dimensions if both the actions of the Wilson and 't Hooft loops scale linearly then both are Higgsed and so there must be massless particles in the spectrum.
Today 't Hooft's classication of phases is the bases of the classification of QCD phase diagram, with the Higgs phase manifested at the cold temperatures and low densities usually found on Earth, massless particles and deconfinment existing at high temperature experiments at RHIC and soon the LHC and perhaps mixed phases existing in the cores of neutron stars.
In 2009, a study by J. Gomis et al., concluded the 't Hooft operator exactly reproduces the results of the dual Wilson loop, proving the predictions.