Last year, I was in a London bookshop chatting to the bookseller about Irish writers when suddenly, Gabriel Byrne appeared. He asked me what I was reading and we had a slightly surreal chat enthusing about writers such as Sara Baume, Eimear McBride and Donal Ryan. And now Byrne’s own book, a memoir called Walking with Ghosts, is published. Only a handful of actors (David Niven, Carrie Fisher, Rupert Everett) make the transition with brilliance. Does Byrne make the cut?

If you’re expecting a vainglorious trot through 40 years of Hollywood, you’ve come to the wrong place. Byrne plays down his achievements, though the book is still shot through with glittering moments. On the phone from his farm in Maine, Byrne, now 70, and with his Dublin accent as strong as ever, explains, “I wanted to write about how you become the person that you become. What influences made you the person you are – your mother, your father, school, the environment you were brought up in, the religion that you either embraced or rejected. It is about the subliminal things that made me become an actor.” He evokes key moments, from first loves and jobs (he was a “dreadful plumber” and a failed priest) to indelibly inked snapshots of violence, shame and loss.

It’s a beautifully judged blend of sparkling anecdotes spliced with the darkest of memories. And Byrne tells a great anecdote. As only the Irish can, he can make something as slight as having indigestion into a funny story; on a short-lived encounter with the nurse, he reflects ruefully, “All we had in common was my appendix”. We hear of mortifying auditions and the disorientating rush of first fame as he starts acting.

Byrne steps back into a childhood where he was the eldest of six growing up in Walkinstown, his mother a nurse and father working in the Guinness Brewery (until he was made redundant). There are wistful memories of the fields, dancehalls and cinema (where he watched ten films a week), now concreted over. The writing has the clarity of a cold Dublin day, and bristles with wit; Byrne proves to be a gifted observer of the “theatre of the street”.

Any rose-tinted nostalgia comes to an abrupt end, and Byrne pulls no punches on his experiences of a Catholic education, until he was expelled and four years in a UK seminary from the age of eleven. He has spoken before about surviving abuse, but admits to feeling vulnerable as the book launches, “because you are revealing parts of yourself”. With a novel you can hide behind your characters, he says, “but with a memoir, on each page you have to speak the truth.”

“That was the difficult part for me writing it, because there’s a lot of stuff in there that I’d prefer not to write about, to be honest. But I felt strongly that there are so many people who have no voice whatsoever, whether children or victims of domestic assault … I felt that by telling my story, which includes abuse and addiction, it brings those topics out into the light. And if people say ‘oh, not this again’, I say ‘yes – this again’. Because it’s not like these things have been dealt with – it’s still going on and it’s going on in silence. By highlighting these subjects we force people to talk about them.”

Passionate about equality and women’s rights, Byrne admires those who’ve had “the tremendous courage to reject oppression and ideas that had a strong grip until very recently”. When he left Ireland in 1983, he points out, “people were still being attacked and killed for being gay … Also it’s only a short time ago that women didn’t have access to abortion – and to me there’s absolutely no argument about a woman having complete control over her body. It also should be a right that we choose how we’re going to die. So while we’ve travelled far, this didn’t happen overnight, and we could not have done it without the bravery of people like Edna O’Brien, John McGahern and David Norris, who spoke up against the establishment.”

At times, his own story is heart-stopping, but not bitter. “You have to find a way to let the person who’s reading it be angry for you, rather than be full of anger yourself. You need to get to the place where you can accept that these things happened and be … not ok with it but just accept them. If you’re still angry, in a way they win.” He’s honest about the subsequent descent into addiction, which followed the abuse and his realisation that “I can’t go on like that”.

There are clearly inner scars. “I think I’ve been pretty truthful about my flaws and about the things that really held me back. I wanted to look at the notion of what success is. It’s not necessarily this big comfortable bed that you climb into and you can just nod off and everything is hunkydory. Success is a conflicting thing for many people. And I wanted to look at how the influences of my childhood brought me to this place where I couldn’t accept that a good thing was happening to me.” He believes that the intertwined Irish educational and religious systems were designed to “curb any sense of individuality”; “I was told, ‘You’ll only be any good for the pick and the shovel’. You were never told you’re a good kid. Instead it was, say this, repeat this, believe this about history, about God … You don’t deserve anything. And this becomes indented in your subconscious.”

Acting was a way of stepping into the light. “I have always found refuge in imagination,” he writes. The book is sprinkled with stardust – Ava Gardner, Richard Burton, Gianni Versace and Richard Harris all make cameo appearances, with Burton, naturally, stealing the show: “We’re not curing cancer. Remember.” Byrne is great company as he learns to ride a horse (and attempt a love scene) wearing a suit of armour, or struggles with a squeaky fake sword on the set of Excalibur. His recall is pin-sharp, down to the colour of the shoes someone was wearing. His tougher memories, you feel, are invisible tattoos burned beneath his skin.

There is restraint at play in Walking with Ghosts; Byrne doesn’t dish the dirt on anyone but himself. “I left out so much. I’ve worked with many great directors and actors so I knew all the stories – but I couldn’t repeat those stories because to me it would feel like a form of betrayal. People are vulnerable on a film set … so I chose not to write that.” His own modesty is disarming. I liked this exchange where tells his father: “Remember when I asked you to come and see me in my first play? And you asked how much it was to get in? I told you five quid and you said, ‘Sure, I can see you at home for nothing’”.

Books have always provided comfort, though at the moment, reading fiction “seems to be either an escape or not equal to reality – what’s happening out in the real world is far more fictional than a book is, somehow”. He admires Raymond Carver, Kent Haruf, Steinbeck, Hemingway, writers who manage a complex simplicity. “And there is of course a whole new brigade of young Irish writers who are fantastic.”

He continues to observe. “I usually write in cafés, in the back of a little notebook. I can’t sit in a room and look at a blank wall, I’d go insane. I have to be out where there are people doing things, and scribble away. Some people have a set structure and I admire that discipline tremendously but I’ve never had that.” He’s currently writing a novel, with “a sense of freedom”. Don’t ever write on an iPad, though, he counsels – he lost 300 pages of an early draft, and found himself pleading at Apple’s Genius Bar: “Look, if the CIA can get stuff out of computers … You’re a Genius, can you really not get it back?”

Having just finished filming in Wales (the second season of War of the Worlds), Byrne sounds relieved to be home, rather than having an early call, or having to “go and fight an alien, or run up and down with a fake gun being chased by things with two heads … I really enjoy time off – so many people prioritise work over life and I always wanted to enjoy my life as much as I enjoy my work.” He’s all too aware of time passing: he tells a poignant story of Laurence Olivier, no less, and conjures another enduring image of his own beloved father’s watch ticking at the bottom of the Venice lagoon. Forget the Emmys, it’s Byrne’s honesty you admire in this moving memoir. The words he tries to live by (Shelley’s) have never been more apt: “Fear not the future/Weep not for the past.”

Walking with Ghosts, €14.99, Picador.