Entanglement entropy in quantum field theory is UV-divergent, which makes it a challenging quantity to analyze from an algebraic perspective. In this talk, I will describe how perturbatively coupling to gravity improves this situation, resulting in a well-defined notion of renormalized entropy in the semiclassical limit. This entropy is constructed using techniques from the theory of von Neumann algebras, and agrees with the generalized entropy of a subregion, consisting of the sum of the quantum field entanglement entropy and the area of the entangling surface. As an application, I will show how to derive the generalized second law for black hole horizons in terms of this renormalized entropy. Time permitting, I will also discuss a construction of a gravitational von Neumann algebra in a slow-roll inflation background, and describe how the background provides an intrinsic notion of a cosmological observer.