There are houses all over the island; but most of them are not occupied this time of year. The folks here fall into a variety of categories: full-time residents (there are maybe 30 of us); snowbirds who live in warmer climates November-April; homeowners who come out for a few days every month year-round; folks who come only in the summer and stay most of the summer; and summer folks who come and go; and oh yes, folks who used to come up a lot but only appear for a few days in summer.
The island is filled with houses that are unoccupied most of the year. When there are lights on, we can not only tell who's here, but often how many people (if a family is here with the kids or grandkids, the lights upstairs and down are on...etc). I can even tell who's here from across the bay, tiny points of light -- "Oh, the so-and-so's are here!"
A place filled with mostly vacation homes is an interesting place in the off-season. The houses aren't abandoned, but they aren't occupied. They are watched after and kept up (by myself and others); but empty. Some houses are dust-sheeted for the season, others look like someone might come home at any minute.
Although folks don't have complete freedom as to what they can build here, most built the house they wanted, which means there are houses of wildly diverging styles are scattered hither and yon. Architect-designed houses with granite counter tops, open plans, wood beams Eames chairs or French Provincial dining tables...compounds with interconnected buildings, "ski-chalets" with huge stone fireplaces left over from the 1960's, airplane hangars with living space, trailers, yurts, stacks of containers on their way to being a dwelling, "freeway houses" bought for a dollar and barged to the island when I-5 was being built; and a hundred variations on the "cabin" theme.
But unless you know where to look, there doesn't look like there's much on the island at all.