Promoting Research Using Diagnostic Codes

During the annual COHRI Winter Meeting which was held in Vancouver, Canada, I had a chance to talk to Dr. Elsbeth Kalenderian, Chair, Oral Health Policy and Epidemiology, Chief of Quality, Harvard Dental Center, Harvard School of Dental Medicine about her role in the development and implementation of the EZCodes dental diagnostic terminology. The EZCodes Dental Diagnostic Terminology was developed in 2009 by a diagnostic terminology research workgroup led by Dr. Kalenderian. Catalyzed by the Consortium for Oral Health Research and Informatics, the EZCodes Terminology has been adopted by 15 dental schools and numerous nonacademic institutions in the US and Europe, creating diagnostic centric care for over 2 million patient visits per year. The EZCodes have been mapped to the CDT and SNOMED terminologies. Recently, the group was just awarded ADA Gies Award “Collaborative pilot study of the impact of the use of the EZCodes dental diagnostic terminology in treatment planning on critical thinking skills of dental students as measured by the Health Science Reasoning Test.” The purpose of this project is to investigate whether or not the introduction and use of the dental diagnostic terminology (EZCodes) in treatment planning per the electronic health record (axiUm) can positively impact dental students’ critical thinking skills. This is a collaborative effort between the Medical University of South Carolina, Harvard and UT Houston.

Please watch the interview:

If you are interested in learning more about the EZCodes just contact Dr. Elsbeth Kalenderian at Harvard School of Dental Medicine.

Associate Dean, Office of Faculty Development and Information Management
Associate Professor, Dental Public Health, Center for Dental Informatics
School of Dental Medicine, University of Pittsburgh

Reflections on MOOCs

I just just completed my first MOOC (massive open online course) “Introduction to Infographics and Data Visualization” where I have learned new techniques for visualizations. As one interested in learning technologies, this experience was not only about gathering new knowledge about infographics, but also about how one handles 2,000 students in an online course.

In this blog, I would like to share some of my personal observations and thoughts about MOOCs and post some questions for discussion:

  • How will this trend influence the students who will enter our dental schools (At Berkeley, one of our students reported, undergrads are encouraged to stay home and watch the lectures instead of going to class)?
  • Is there an opportunity for dental educators to move some of the preclinical classes to an online delivery mode (especially when you lack the basic science faculty to teach them)?
  • Can information about student behavior collected during online classes support research in the realm of Scholarship of Teaching and Learning?
  • Does any of you have any experience with courses delivered through Udacity, Coursera or edX?

My first observation has to do with motivation: You have to be a self-motivated, self-disciplined individual to finish a MOOC. There is very little external pressure to go on, especially peer pressure is low in an online environment. We all have experienced that online communication works best when you interact with people whom you have previously met face to face. Well, I did not know any of my 2,000 peers.

One way to reduce the dropout rate is to engage students which brings me to my second observation which I made during the course participation. The course director, Alberto Cairo who teaches Information Graphics and Visualization at the School of Communication at the University of Miami, made great efforts to keep us, online students, attentive and motivated. He used many techniques of peer review and peer rating of project (not grading!) to facilitate this process. In essence, each assignment included a mandatory part of posting at least two comments. In addition, he gave us the freedom to do whatever we want for our final project instead of a very prescribed assignments. He had us choose a topic, gather the appropriate information, and present the idea of how to show that information in graphic form. The first part of the assignment was to do sketches and write a short description outlining our goals and then share them in the discussion forums to get feedback from peers. We were also supposed to comment on other people’s proposals. Then, we had two weeks to produce our infographic and post it—again the assignment included to comment on the projects of our peers. While I did not check out all submissions, from the ones I reviewed, I can say that many students picked a topic which was either related to their daytime job or otherwise close to their heart. So, they were not only interested in the visualization techniques, but also felt passionate about the topic—I believe that made a difference.

Nicholas Carr, the author of The Shallows: What the Internet Is Doing to Our Brains (, published an interesting piece about MOOCs in Technology Review addressing, among other aspects, the dropout issue (see this and other links at the end). The article explores historical analogies, like the correspondence courses of the 1920ies, and talks about newer concepts, like the flipping of the classroom Khan Academy style. In general, the discussion centers around the efficiencies of MOOCs versus the lack of meaningful interactions between students and teachers resulting in high dropout rates. “Scholars who are skeptical of MOOCs warn that the essence of a college education lies in the subtle interplay between students and teachers that cannot be simulated by machines, no matter how sophisticated the programming.”

Yes, I will get a certificate, but “no formal course credit of any kind is associated with the certificate.” This brings me to my next observation, the question of proctoring technologies in these MOOCs. I do not have to elaborate on the issue of cheating in an environment where everyone is alone in front of a computer. I do think that both remote and face-to-face proctoring business is going to grow because of these new course modalities. Technology Review recently featured an article about upcoming technologies in that area: “In Online Exams, Big Brother Will Be Watching. How can you tell if an online student has done the work? That’s where webcam proctoring comes in.”
I have chronicled my journey during this MOOC experience in my blog if you are interested to learn more:

Let me conclude with a quote by Nicholas Carr: “Whether massive open courses live up to their hype or not, they will force college administrators and professors to reconsider many of their assumptions about the form and meaning of teaching. For better or worse, the Net’s disruptive forces have arrived at the gates of academia.”

MOOC Resources
NYT article about MOOCs:
Nicholas Carr:
The Big Three, at a Glance:
In Online Exams, Big Brother Will Be Watching:

Associate Dean, Office of Faculty Development and Information Management
Associate Professor, Dental Public Health, Center for Dental Informatics
School of Dental Medicine, University of Pittsburgh

Report from the COHRI Summer Meeting

Hi everyone,
I would like to report about the recent COHRI Summer Meeting that was held in Boston this year—co-hosted by the Tufts University School of Dental Medicine and the Harvard School of Dental Medicine.
Informational Brochure:
COHRI’s site:

The Consortium for Oral Health Research & Informatics (COHRI) will:
•     Create, standardize and integrate data using electronic health records.
•     Cooperate with other health related institutions to share data.
•     Improve informatics utilization in dental education, health care, and research.
•     Work together as a consortium to develop research projects to promote evidence based dentistry.
•     Define and facilitate the implementation of best practices and standards of care.

One of COHRI’s projects is the Dental Data Repository (DDR) which houses 1.1 million patient records–pooled from four dental schools (limited dataset, de-identified for HIPAA reasons) using I2b2. The DDR can accept and integrate data from disparate dental data sources, and allow end users to explore and extract information to support their specific research or decision making needs. For instance, researchers can query the data on the Website and ask questions like: give me all females with caries risk “high,” give me the race distribution for the result set. The results will be instantly display on Website (in numbers only). After obtaining the appropriate IRB approval and approval by the COHRI Project Review Committee one can get the full data set in structured format. Please contact Dr. Muhammad F Walji for more information:

I had the privilege to chair this year’s COHRI Education Steering Committee meeting. Our group established consensus that there needs to be more focus on ways to improve the learning outcomes of our students with the help of the EHR. As it stands now, we are somewhat limited in our approach as we facilitate education by keeping track of grades and credits for procedures. For instance, we asked how we efficiently could drill down into our treatment outcomes data and find out what students struggle with most; or how do we promote teaching beyond what we could previously do with paper records?

Our vision for technology-based clinical teaching is using existing technology to improve clinical teaching by applying methods that have been proven successful in the education sciences, like the use of scaffolding techniques for students and clinical instructors. The group felt that the time is ripe for some enhanced approaches since many schools have completed the EHR setup and solved the important clinical and financial feature implementation concerns.

The Education Steering Committee will focus on:

1. E-portfolios
The group felt that e-portfolios would not only serve to help students see what they have accomplished, but also improve their chances when applying for residencies as the National Boards will convert pass/fail eliminating their use for applicant screening. Contact me if you are interested in contributing:

2. Virtual Case Exchange
One of the questions raised was how do we manage to share interesting cases across students, faculty and residents inside our schools and across COHRI member schools.
And, how can we provide easy-to-access virtual cases that would support clinical teaching with technology to make our overworked clinician-educators more effective.

3. Pool Training Resources Across Schools
Many schools create axiUm training resources for their internal use, potentially causing duplication of work across schools. We plan to share these training resources under a CC license which allows schools to modify the content (with attribution). If you are interested in working on this project, please contact Eric Salmon: esalmon@PACIFIC.EDU.

Associate Dean, Office of Faculty Development and Information Management
Associate Professor, Dental Public Health, Center for Dental Informatics
School of Dental Medicine, University of Pittsburgh

Information Technology in Academic Medicine

Hi everyone,

Today, I am reporting live from the 2012 Information Technology in Academic Medicine Conference Sponsored by the AAMC Group on Information Resources (GIR) taking place in Austin, TX.
Event Website:
GIR’s site:
While, I will present here about our recent e-learning development my focus is on learning how medicine is using information technology in education. The GIR “provides a forum for individuals in relevant roles of leadership and responsibility to promote excellence in the application and integration of information resources in academic medicine, including medical education, clinical care, and health sciences research.”

So, what was really interesting? First, I attended today’s plenary session which featured Dr. Thomas Talbot, AFSIM Chief Scientist at the Telemedicine and Advanced Technology Research Center whose presentation was titled “Designing Medical Education for Today’s Brain.” His charge was to think about how a medical school should educate students, disregarding the status quo. His intriguing talk started off with outlining what has changed in the recent past: (1) more and better technology is available, (2) our knowledge about how students learn has changed, (3) our students’ expectations have increased when it comes to technology use, and he pointed out that (4) residency training has dramatically changed. Based on that premise he came up with some thought provoking ideas, such as the notion that the problem of teaching in medicine is not about comprehension, because most students are pretty smart and can learn by themselves, but that it is an issue of volume. Thus, we need to ask the question of how can we make education more efficient. Can we teach more through optimization? And, as we have only limited time, can we spend the time with the students meaningful. Lecture time does not count as meaningful in his view.
On a different note, he challenged the usefulness of gatekeeper courses? He claims that organic chemistry is really not necessary to become a successful doctor, but that it is mainly used as a gatekeeper to check the intellectual abilities of students. He suggested that the time might be ripe for pushing useful courses into the undergrad education instead of wasting time with gatekeepers.
He also talked about what he calls the “two sets of books problem.” To illustrate this concept he explained a personal anecdote: When he entered medical school he bought for a course an expensive book and he also purchased notes from previous years. Then, he went to all lectures to figure out which topics form the notes are relevant. At the end, he was able to use the copious primary and his secondary notes to come up with a study plan. At the end he found the book quite helpful from which he mostly studied. However, his test score was significantly lower than that of his friend who just used all the time studying the textbook, never went to any lecture and did not purchased any notes. I am sure something like this can only happen in a medical school, but never in a dental school. 😉

The second interesting talk today was about 3D visualization delivered by Dr. Charles Goodacre, Dean of Loma Linda University School of Dentistry “The Combined Use of Dynamic 3D and Static 2D Visualization to Enhance Student Learning.”
Dr. Goodacre reported about his work at Loma Linda in collaboration with faculty from Stanford in how we can best enhance student learning through computer visualizations. He demonstrated several programs including products from eHuman which incorporate 3D resources and animations. He also referred to the use of quizzes with reference libraries and games that assess speed and accuracy in 3D environment.

While computer-based visualizations have become increasingly popular in education, they might not always reflect students’ learning preferences. He pointed out that most but not all studies indicate that students prefer to learn using multiple modes rather than a unimodal approach. For instance, one study reported that 56% of students have a preference for multimodal learning, but we still use overwhelmingly lectures in dental education. I was impressed by his command of the current literature in cognitive science and how he explained complex concepts like cognitive load theory by Sweller, specifically how intrinsic and extraneous load play a role in 3D simulations. Or, how concept from Mayer regarding how the learner’s control over the information flow can deepen the understanding of the topic apply to computer-based visualizations.

Here his key thesis from the summary slide:

  • Interactive learning is preferred by most but not all students.
  • Interactivity should include opportunities for feedback and self-assessment.
  • High Visualization Ability (VZ) students benefit by using dynamic 3D visualizations.
  • Low VZ students do better with static visuals than dynamic 3D visualizations.
  • Interactive 3D visualizations needs to be demonstrated by faculty in class to show their effective.
  • Computer programs need to have both static and dynamic 3D visuals to meet the needs of as many students as possible.
  • Visuals and text should appear together.
  • Information should be presented sequentially 
rather than all at once.
  • Visuals should have labels that can be hidden or revealed.
  • Animations should be slow enough for learners to observe changes; there should be interactive controls.

Got interested? The next GIR meeting will be in Vancouver, BC June 5-7, 2013.

Associate Dean, Office of Faculty Development and Information Management
Associate Professor, Dental Public Health, Center for Dental Informatics
School of Dental Medicine, University of Pittsburgh

What should guide our adoption of emerging technologies into the dental educational process?

In a widely discussed 2008 Atlantic Monthly article, Nicholas Carr asked, “Is Google Making Us Stupid?”. His answer describes a faltering ability to concentrate and the suspicion that the Internet is a key contributor. “My mind now expects to take in information the way the Net distributes it, in a swiftly moving stream of particles. Once I was a scuba diver in a sea of words. Now I zip along the surface like a guy on a Jet Ski.” Our current students belong to a generation which has been growing up with ubiquitous access to online information and are often referred to as “digital natives” or the “Google generation.” Many claims have been made that this generation process information differently (e.g. The Shallows by N. Carr) going so far as to assert that excessive online “browsing” results in anatomical and physiological changes in the brain (1,2). While we could join the discussion which laments about the perceived or real loss of sustained attention among the members of this generation; or about the inability of many of our students to point out the building on campus which houses the library; or our students’ Facebook obsession; we might easily fall victim to the same fallacy as earlier generation did. Many of our ancestors have resisted the introduction new technologies ranging from the introduction of trains which were supposed to kill all horses to the invention of TV which was perceived as the end of radio. “Intellectual technologies,” have always been vehemently criticized: Socrates thought that the introduction of writing would substitute the knowledge people used to carry inside their heads (3); or the easy availability of books through the arrival of Gutenberg’s printing press would lead to intellectual laziness, making men “less studious” and weakening their minds (4). Instead we should embrace such intellectual technologies as they can enhance our understanding of the world, “consider how maps and clocks have altered our relation to space and time, developing in us a more abstract sense of the measurement and order of both” (5). We, as dental educators, should initiate a productive discourse about how we might need to adjust our teaching style. How do we modulate the relationship of technology, teaching and the generation of our students? How we deliver information is not just a passive act of transmission–media supply the stuff of thought, but they also shape the process of thought (6).

Food for thoughts….




1) Small G, Vorgan G.  iBrain:  Surviving the Technological Alteration of the Modern Mind.  HarperCollins, New York, 2008.

2) Small GW, Moody TD, Siddarth P, Bookheimer SY.  Your brain on Google:  Patterns of cerebral activation during Internet searching.  American Journal of Geriatric Psychiatry 2009;17:116

3) Plato’s Phaedrus

4) Italian humanist Hieronimo Squarciafico

5) The Shallows by N. Carr

6) Media theorist Marshall McLuhan

Could it be a matter of perspective? “Educating” vs. “Training” in Dental Schools

Hello. I am starting my first DIOC blog entry mainly as a result of communicating with Heiko  recently on measurement issues in dental education. Heiko wrote in ADEA Faculty Development Listserve: “While I do not want to digress too much from faculty development, I would like to add that the one problem we often overlook is the fact that what we define as learning outcomes is not entirely “on target.” While most of us think of dental education as “education,” it is much closer to “training” as we are all working in professional schools whose mission is to produce competent dental practitioners.  While a great dental education goes a long way on the path to becoming a successful dental practitioner, we need to be careful to think of them as exactly the same.“

In general I agree with Heiko’s careful dissection on the issue regarding measuring the outcomes of dental schools. However, I am curious to learn more what your and other dental educators’ takes are, specifically on why most dental educators, including Heiko, think dental schools are more training than educating.

Here is a bit background on why this question emerged. My first degree was physics. 13 years ago, at the end of my graduate study in Curriculum and Instruction, I started working in the field of teacher education and teacher professional development. At that time, I was warned not to use “training” as a word when interacting with teachers and future teachers. It was not obvious to me then why such a word can lead to difficulty and sometimes resentment. Over the years, I have realized how changing the word “training” to “preparation” has helped me reframe my own work with teachers and future teachers. It also helped me to rethink what outcomes I would like to achieve in my work with teachers. The belief of “I know it better than the people I work with” seems to dissipate the moment I took the position that I am there to help prepare future teachers and to enhance current teaching practice. Additionally, “training” seems to be more or less associated with rule-based skills. Yet, teaching is not composed of a set of rule-based technical skills. One can not buy a book on “teaching for dummies” and expect to become an effective teacher overnight. Along with Lee Shuman (2004) and many others, I believe that teaching requires one to develop practical wisdom, of which is nearly un-trainable due to its context-specific nature and practical wisdom is not rule-based, as Aristotle pointed out long ago. I admit that this belief leads to the acceptance (Schwartz & Sharpe, 2011) of how difficult it is to measure the outcomes of teacher preparation programs. For teachers are only getting better the more they practice (if they are reflective).

So, now, let’s switch our focus back to dental education. From the perspective of a dental educator, what difference will it make for you to shift your belief and action from “training” future dentists to “educating”?


Shulman, L. S.  (2004).  The wisdom of practice: Essays on teaching, learning, and learning to teach.  S. Wilson (Ed.)  San Francisco: Jossey-Bass, Inc.

Schwartz, B. & Sharpe, K. (2011). Practical wisdom: the right way to do the right thing. Riverhead Trade.

Is the NMC Horizon Report reliable?

Most of you are probably aware of the release of the new Horizon Report for Higher Education; if not, consider reading it:

“The ninth edition describes annual findings from the NMC Horizon Project, a decade-long research project designed to identify and describe emerging technologies likely to have an impact on learning, teaching, and creative inquiry in higher education. Six emerging technologies are identified across three adoption horizons over the next one to five years, as well as key trends and challenges expected to continue over the same period, giving campus leaders and practitioners a valuable guide for strategic technology planning.”

The report reveals technological metatrends and predicts:

  1. People expect to be able to work, learn, and study whenever and wherever they want to.
  2. The technologies we use are increasingly cloud-based, and our notions of IT support are decentralized.
  3. The world of work is increasingly collaborative, driving changes in the way student projects are structured.
  4. The abundance of resources and relationships made easily accessible via the Internet is increasingly challenging us to revisit our roles as educators.
  5. Education paradigms are shifting to include online learning, hybrid learning and collaborative models.
  6. There is a new emphasis in the classroom on more challenge-based and active learning.
  • Time-to-Adoption Horizon: One Year or Less

– Mobile Apps
– Tablet Computing

  • Time-to-Adoption Horizon: Two to Three Years

– Game-Based Learning
– Learning Analytics

  • Time-to-Adoption Horizon: Four to Five Years

– Gesture-Based Computing
– Internet of Things

When reading such predictions, I am asking myself how reliable are they.  Recently, Martin et al. tried to answer exactly this question in a paper* published in Computers & Education. The authors looked at all reports which have been published since 2004 (they received “more than 500,000 downloads a year and have an estimate readership of about 1 million in 75 countries”).  Martin et al. used “bibliometric analysis which technologies were successful and became a regular part of education systems, which ones failed to have the predicted impact and why, and the shape of technology flows in recent years.” The paper includes several very interesting visuals on how technologies most likely to have an impact on education. The authors conclude: “The bibliometric analysis over the predictions highlights that some of the predictions were right, e.g., social networks, user-created content, games, virtual worlds and mobile devices. Other predictions did not have the expected impact, e.g., knowledge Web, learning objects and open content, context-awareness and ubiquitous computing. However, other predictions were successful, although their impact was delayed one or two years, e.g., grassroots videos and collaborative Web.”

Do you think that the Horizon Report trends have an impact on how you evaluate technology for dental education?



* New technology trends in education: Seven years of forecasts and convergence by Sergio Martin, Gabriel Diaz, Elio Sancristobal, Rosario Gil, Manuel Castro, Juan Peire. Computers & Education (2011). Volume: 57, Issue: 3, Pages: 1893-1906

On Dental Education: Should we exploit technology to cater to a generation of hyper-attention learners?

Hi everybody,
I am planning to periodically post topics related to the use (and misuse) of instructional technology in dental education. Here the first installment:

I just read a thought-provoking article “Hyper and Deep Attention: The Generational Divide in Cognitive Modes” by Katherine Hayles which discusses the generational shift in cognitive styles that poses challenges to education at all levels (published here, full text available here).
Dr. Hayles discusses the dichotomy between deep attention, concentrating on a single object for long periods ignoring outside stimuli, and hyper attention, switching focus rapidly among different tasks. The latter is characterized by preferring multiple information streams, seeking a high level of stimulation, and having a low tolerance for boredom.
While many people, in my age group anyway, would immediately argue that deep attention is better, “it comes at the price of environmental alertness and flexibility of response.” Dr. Hayles argues that “Hyper attention excels at negotiating rapidly changing environments in which multiple foci compete for attention, …”  However, in our traditional educational environment “hyper attention [is] regarded as defective behavior.” The problem we are facing is, according to Hayles, the clash between the “expectations of educators, who are trained in deep attention […] and the preferred cognitive mode of young people.”
Her research also shows that while the “mean [of the population] moves toward hyper attention rather than deep attention, compensatory tactics are employed to retain the benefits of deep attention through the artificial means of chemical intervention in cortical functioning,” such as through cortical stimulants (e.g. Ritalin). Her article elaborates on research about synaptogenesis which is altered when children grow up in media-rich environment–reminding me of The Shallows by Nicholas Carr.
One of her key arguments is that “A case can be made that hyper attention is more adaptive than deep attention for many situations in contemporary developed societies.” However, I would note that the public thinks differently when it comes to the work of health care providers, see: New York Times: As Doctors Use More Devices, Potential for Distraction Grows, and AHRQ recently reported about a “multitasking mishap” in their Morbidity & Mortality Rounds on the Web.
CODA, the accrediting body for all US dental schools, states in its standards that the “[u]se of technology in dental education programs can support learning in different ways, including self-directed, distance and asynchronous learning.”

What challenges are we facing when these students enter our dental schools? Do we want to foster hyper attention, building on their acquired predisposition, or do we want to change their cognitive style to adopt a style of deep attention which is more suitable for a health care provider? Actually, is deep attention really more suitable for health care providers who must interact with often multiple computers and devices as well as need to adjust to the rapid pace of many patient encounters per day?
Do we need to prepare our dental educators for these hyper-attention learners? At the University of Southern California, researchers “explore new pedagogical models that provide greater stimulation than the typical classroom […] 14 large screens span the walls, providing display space for […] participants [who] search the Web for appropriate content to display on the screens while a speaker is making a presentation.” Think about dental educators: do you think they would enjoy teaching under such circumstances?
Or, maybe the solution can be found in what Atul Gawande recently wrote in The New Yorker regarding the promises of technology:  “What ultimately makes the difference is how well people use technology. We have devoted disastrously little attention to fostering those abilities.”
Heiko, looking forward to your comments and thoughts

Associate Dean, Office of Faculty Development and Information Management
Associate Professor, Dental Public Health, Center for Dental Informatics
School of Dental Medicine, University of Pittsburgh