Daniel Nicoletta was 19 when he pitched up in San Francisco, young, gay and racked with self-loathing. It was 1974, the Castro was in the process of becoming the LGBT mecca it is today, and the apartment Nicoletta rented was one block up from Castro Camera, the shop and campaign headquarters of Harvey Milk, who would go on to become one of the first openly gay people in the US to be elected to public office. Nicoletta took his film there to be developed and ended up finding his tribe, his political consciousness and his vocation.

“They were super gregarious,” recalls Nicoletta of Milk and his partner, Scott Smith. “I was enchanted with how friendly they were.” So enchanted, in fact, that he didn’t realise he was being cruised. “I was new to all that,” Nicoletta laughs. “I had grown up in an environment that was not pro-gay. There was a lot of confusion around Catholicism. I was pretty vulnerable. Suddenly I was in this kissing and hugging fest, surrounded by people who were in the process of creating a better world for LGBT people. I quickly discarded the self-loathing.”

A year later, Nicoletta was working in the shop and honing his craft as a photographer. He documented Milk’s struggle to be elected to public office, taking many of the famous shots of the man he describes as his mentor and gay parent. The most recognisable photo of Milk – outside Castro Camera, all trademark saucy grin and flapping tie – is one of his. Nicoletta was there in the grief- and rage-stricken aftermath of Milk’s assassination by fellow city supervisor Dan White a year later. Ever since, he has continued to point his lens at his community.

More than 40 years later, the first collection of Nicoletta’s photographs is finally being published. LGBT: San Francisco is a joyous, poignant and occasionally sombre record of the city’s lesbian, gay, bisexual and trans people, taken from an extraordinary archive. It is at once a celebration, an exercise in visibility and a timely reminder of how recently the battles for gay rights were won and how fragile those rights remain. It is also a call to action. “The message of the book is that people should not give up hope,” Nicoletta says. “And the way to manifest hope is to take action.”

There doesn’t seem to be an event, cultural flashpoint, protest, riot or handlebar moustache that Nicoletta’s lens hasn’t captured over four decades in the Castro. Like all great reportage, his work tells the story of an entire movement. There are photos of Milk campaigning, alongside stills from Gus Van Sant’s biopic, Milk, on which Nicoletta worked as set photographer, consultant and actor. Images from countless Castro street fairs and Pride marches, of drag queens and kings, cruising and Armistead book launches, radical faeries and queercore artists, White Night riots and Aids vigils, anti-censorship demos and equal-marriage protests. Sylvester, Divine, Grace Jones and Allen Ginsberg are all here, but so too are hundreds of unknown and otherwise unsung LGBT people.

Does Nicoletta have a favourite period or photo? “I love the one of Harvey dressed as a clown by the sea,” he says referring to a monochrome shot taken in May 1978, just six months before Milk was murdered. “We were at a mental health fundraiser, saw these hang gliders and decided to walk out there and cruise them. The picture just happened. Harvey was being the ham that he always was and I got the lucky shot. It’s my gift from Harvey.”

How drastically has the Castro changed since he started photographing it? “It’s more difficult for radicalism to survive in San Francisco now,” Nicoletta says. “But it is there. You can’t survive on a dime like we did in the 70s, but the creation of a ghetto for LGBT people is a core thing. It will always be a place of pilgrimage. For a young gay kid getting off the bus from Newark, it could still be magnificent. And young people are galvanising in San Francisco again. The city is relentless. It’s not going to take [the age of Trump] lying down.”

Nicoletta has changed, too. He is no longer “on the frontline” of documenting LGBT life. “I just shot the Mermaid Parade in New York and it killed me,” he says. “My back is on fire today.” Three years ago, he left San Francisco for rural Oregon and what he describes as a “monastic life” with his longterm partner. So what is he photographing now? “Well, it turns out you can’t hide from your history,” he laughs. “The rural queer community has found me. I shot a queer forestry camp recently and it was one of the best days of my life.”