Luckily, our oldest has enjoyed looking at books. He has always had a collection that he has had easy access to. He's been able to look at them and feel them whenever he wanted to; with an adult there or by himself. He's been able to turn the pages, and feel the fabric books, point at pictures and talk about the pictures. These books have been so handy as a quiet activity to take outside of the house. If you have to take your toddler to a meeting, appointment, ceremony or some other place where they need to be entertained quietly, books are the first choice.
Keeping books easily accessible for your child means that you can show them how you pack up. They can see how we store books on the shelf, standing up with the spine facing outwards. They will learn the vocabulary words for these activities. If you want to take some books on an outing, you can recruit them to help you choose books and ask them to put them in the bag you have out. These activities help boost their self esteem, learning to prepare and language development.
Eventually your toddler will grow up as has mine! So keeping age-appropriate books out and easily accessible is one thing you can consistently do to help your child like books and reading.
So I encourage you to keep your toddler books for when your toddler is old enough to read. They will want to read the books you have, which is good practise, and it will give them confidence.