We have social norms against bugging our houseguests, but we have no social norms that require us to explain to them that there'll be a record of every light that they turn on or off. This feels like it's going to end badly.
That's some very interesting observation. Another thing I have pondered is, what are my rights as a visitor if I visit someone with an Amazon Echo or Samsung TV that's always listened. When installing these things into your own home you implicitly or explicitly accept their terms for listening on you, but as a visitor, I never agreed to any such terms.