SEO is the science of customizing elements of your web site to achieve the best possible search engine ranking. That is really all there to search engine optimization. Both internal and external elements of the site affect the way it's ranked in any given search engine, so all of these elements should be taken into consideration. Good SEO can be very difficult to achieve, and great SEO seems pretty well impossible at times.
Ideally you want your site to be displayed somewhere on the first three pages of results. Most people will not look beyond the third page, if they get even that far. The fact is, it's the sites that fall on the first page of results that get the most traffic, and traffic is translated into revenue, which is the ultimate goal of search engine optimization.
To achieve a high position in search results, your site must be more than simply recognizable by a search engine crawler. It must satisfy a set of criteria that not only gets the site cataloged, but can also get it cataloged above most (if not all) of the other sites that fall into that category or topic.
Some of the criteria by which a search engine crawler determines the rank your site should have in a set of results include:
Anchor text
Site popularity
Link context
Topical links
Title tags
Keywords
Site language
Content
Site maturity
Some of the criteria listed also have multiple points of view. For example, when looking at link context, a crawler might take into consideration where the link is located on the page, what text surrounds it, and where it leads to or from. For some search engines, links are more important than site maturity, and for others, links have little importance. These weights and measures are constantly changing, so even trying to guess what is most important at any given time is a pointless exercise.
By nature, many of these elements are likely to have some impact on a site's ranking. However, without your attention, you are leaving the search ranking of your site to chance.