The time travel thing just annoys me so much, because I absolutely hate it as a plot device, and it changes my perception of the weirwood net into something that just isn't as interesting to me. (Okay, this is going to be more of a rant on how it impacts the books if this element plays out there as it did in the show)
In hindsight maybe this was inevitable that ideas could be warged across time and space... Jojen and Bran both have had prophetic (if more symbolic) dreams for quite a while. Bran's dreams are probably more of the "3ER is talking to me from a great distance" and have no time travel (he basically dreams of Ned's death, as does Rickon, before they know about it, but that's before the raven arrives with the news, not before it actually happens). More significantly, Jojen claims to know how he's going to die, and foresees things like the Ironborn taking Winterfell.
But that's not quite how I had thought the weirwood net (namely Bran's visions from ADWD) were presented. The way that was presented made it seem more like the pensieve in Harry Potter, or the neural network in Avatar: the weirwoods were like this database of memories from the past (as Bran does when starts seeing some visions), and are able to produce a sort of hivemind where you could communicate from one weirwood to another (like when Bran whispers to Theon). As such, the memories/history are just like rewatching an .mp4 file through trees that also function as a sort of phone system.
The idea that you are ACTUALLY being taken back to that event in history does a few things. First, it means you aren't seeing a memory of Tower of Joy, you are actually at the Tower of Joy. This raises interesting questions, like "why are Bran and 3ER watching TOJ alone? Is this the only time they show up to watch it? Does no one else with this power ever visit here? Why aren't there, at the least, one or two other copies of Bran from when he might want to revisit this to learn more, or of 3ER from any time he might have watched this before?". I mean, its not like you can't have multiple beings in a "memory" at the same time, otherwise I have no earthly idea what was going on with the Nights King.
Second, it means that we now expose ourselves to the singular reason I hate time travel: paradoxes. If you can travel back in time (or even send an idea back in time), why hasn't Future Bran decided to do what 3ER completely forgot to do and warn his past self about "oh, hey, try not to spelunking by yourself, or at least avoid White Walkers if you do"? It's Magic TV Logic and/or TV Characters Behaving Like Idiots at its worst if you don't explore this. Granted, the show (and books?) might very well subvert this. The DeLorean can't return from the 1800s if it's out of fuel, maybe there's something that stops Bran from being able to use this power again. That'd be fine, but then it just makes the time travel into a one time thing, which sort of makes the weirwoods into a giant red herring and/or shaggy dog story. Dumb.
Really, the only way out of that pickle (no one uses time travel, even though its totally still an option) is
the Red Vs. Blue S3 approach: you never actually change anything through time travel, you just either end up causing the events that transpired or you have no impact whatsoever. Which is probably the best way to deal with the time travel paradox, but probably is better suited for a comedy than whatever we're trying to accomplish here.
TL;DR version: Time travel is ****ing stupid and brings up more questions than it answers.
